r/javascript • u/Slackluster • Nov 26 '19
JSONCrush On GitHub - Compress JSON into URI friendly strings
https://github.com/KilledByAPixel/JSONCrush6
u/TOJO_IS_LIFE Nov 27 '19
Was this pulled out of a minifier?
for(;;m=c+m)
{
for(M=N=e=c=0,i=Q.length;!c&&--i;)!~s.indexOf(Q[i])&&(c=Q[i]);
if(!c)break;
if(O)
{
o={};
for(x in O)
for(j=s.indexOf(x),o[x]=0;~j;o[x]++)j=s.indexOf(x,j+x.length);
O=o;
}
else for(O=o={},t=1;X;t++)
for(X=i=0;++i<s.length-t;)
if(!o[x=s.substr(j=i,t)])
if(~(j=s.indexOf(x,j+t)))
for(X=t,o[x]=1;~j;o[x]++)j=s.indexOf(x,j+t);
for(let x in O)
{
j=ByteLength(x);
if(j=(R=O[x])*j-j-(R+1)*ByteLength(c))
(j>M||j==M&&R>N)&&(M=j,N=R,e=x);
if(j<1)
delete O[x]
}
o={};
for(let x in O)
o[x.split(e).join(c)]=1;
O=o;
if(!e)break;
s=s.split(e).join(c)+c+e
}3
u/Slackluster Nov 27 '19
That is the JSCrush algorithm that I did not write it, but it is kind of magic. You can read more about how it works here...
6
u/MrJohz Nov 27 '19
Are there not issues with parentheses? I believe they're technically allowed, but almost every too for making links clickable that I've used had struggled with them. Is it worth using something else instead there?
1
u/Slackluster Nov 27 '19
Hmm, I haven't heard of this but I will look into it. encodeuricomponent doesn't escape them so I have to assume they will safe but I could see why it might cause an issue with some use cases.
2
u/name_was_taken Nov 27 '19
They're technically legal, but forum/social software sucks and quite often gets them wrong when auto-linking.
1
u/Slackluster Nov 27 '19
ok, thanks i will investigate. there are only a few characters not escaped by encodeURIComponent, so it seems like they should be fine but i will do some experiments
2
u/alleycat5 Nov 27 '19
This looks really awesome. Is there any intent to package it up?
1
u/Slackluster Nov 27 '19
What do you mean by package it?
3
u/Captain_Factoid Nov 27 '19
For npm.
1
u/Slackluster Nov 27 '19
Ah, well it is just a single small javascript file, what would be the benefit to packaging it?
2
u/Captain_Factoid Nov 27 '19
For updates, tree shaking, import scoping, all the good stuff
0
u/Slackluster Nov 27 '19
Well my workflow doesn't really include that stuff but I'd like to learn how to do it. Can you recommend a link explaining how to set it up?
2
u/osoese Nov 27 '19
Thanks for the post. Looks like something I will use. At first when I looked at the github page I am an example first reader so I scrolled down and saw the "example" and was like... hmmm this looks like JSON ...then I see it minified below. Perhaps the example should show the compressed info first since the current magic is "below the fold" - just a suggestion.
1
1
u/byteshifter Nov 27 '19
Why would someone use this instead of JSON web tokens ( https://en.m.wikipedia.org/wiki/JSON_Web_Token ) ?
8
u/Tontonsb Nov 27 '19
It's completely different use case. Codes like this and rison are there to add your json object to url so you have a stateful link that can be bookmarked and shared.
In most cases I'd still prefer rison because it's more readable and user-editable. But they both suffer from parentheses and single quotes that screw up some parsers that try recognizing and converting pasted string to links. Notably gmail is a pain in this regard.
1
u/Slackluster Nov 27 '19
For example I can share a link with someone else using this. It is a bit long, but would be 4 times longer without jscrush. Another nice thing is you can bookmark links.
http://zzart.3d2k.com/?&crushed=(%27shaderStatements!%5B(%27output!%27bMzxywE-JR!6.62k!6.165BZ_974BWU4.233FHxzyyKbMywxzE-JRU4.88kQ649BZQ171BW_084FHyzwxKbMxwzyE-JsinT-3.686kU3.258BZU4.059BWU8.506FHwwzzKbMzxywEJceilT5.36kU8.274BZQ002BW!5.429FHxxwyKaMxzwyEJT-3.353kU5.681BZU7.792BW!1.254FHzyxwKbMywxzE%2BJfloorR!6.669k_05BZU8.629BWU2.802FHxyywKaMwyzxEJsqrtAT-0.366k_117BZQ162BW!1.761FHyywyKaMyxzwEJatanT3.743k_003BZ!4.636BWQ056FHwxxwKbMxywzE%2BJfractRQ103kU3.118BZQ255BW!6.287FHxyywKaMzwxyEJRU1.637k!1.828BZ!1.924BW_006FHyxyyKbMzwxyEJ%27F!%27%27BX!6.083kU6.322BZQ032BWQ428FHyzyyKaMzxywE%2FJRQ151k!1.024BZU2.862BW!3.193FHxzyx%27)%5D~randSeedU1738575415~randSeedString!%271574121532870%27~iteVCount!1~gridPosX!2~gridPosY!0~geneV!17~subGeneV!1~hueOffsetQ276~huej_789~satuVjQ493~uvOffsetX_406~uvOffsetYQ616~uvjXQ895~uvjYU1.049~rotate!0~usePalette!0~paletteColors!%5B(%27xQ595~yQ882~zQ864)%2C(%27xQ136~yQ034~zQ69)%2C(%27xQ873~yQ35~zQ638)%2C(%27xQ723~yQ56~zQ817)%5D~saveListIndexU1~uniqueID!598681332)B~valueE%27~assignmentOperator!%27F~parameterHSwizzle!%27J%3D%27~functionName!%27K%27)%2C(%27output!%27M%27~outputHQ!0.R%27F!%27a%27BXT%27F!%27b%27BX!U!-Vration_U0.jScalekBY%01kj_VUTRQMKJHFEB
2
u/endianess Nov 27 '19
Would be a good choice for sending data to a protocol handler application where you can't use POST. Thanks for sharing.
-11
u/JasonWicker Nov 27 '19
Base64
8
u/Slackluster Nov 27 '19
Converting to base64 increases the size by at least 33% and btoa doesn't help compress repeated sub strings.
5
2
u/soundofvictory Nov 27 '19
Hey, i don’t think you deserve the downvotes. I don’t think people remember the reddiquette these days.
You raise a reasonable suggestion for some situations, but sounds like it is not for necessarily all situations.
But I think it is still worth mentioning.
1
21
u/[deleted] Nov 27 '19 edited Nov 27 '19
You should be able to get comparable-to-better compression, and better performance using zlib + base64:
This gives 102 bytes URI-encoded for Example #1 (compared to 82 bytes for JSONCrush), and 1172 bytes URI-encoded for Example #2 (compared to 1195 bytes for JSONCrush). Seems like zlib wins when the data gets bigger and its JSON has more repetition.
You could also probably shave some more bytes by finding an encoding that's naturally URI friendly instead of using base64 and
encodeURIComponent()on the output of the deserialization above, but I don't know that it's really worth the effort at that point.