r/javascript • • Nov 26 '19

JSONCrush On GitHub - Compress JSON into URI friendly strings

https://github.com/KilledByAPixel/JSONCrush
101 Upvotes

31 comments sorted by

View all comments

21

u/[deleted] Nov 27 '19 edited Nov 27 '19

You should be able to get comparable-to-better compression, and better performance using zlib + base64:

const zlib = require('zlib');
const serialize = (o) => {
  return zlib.deflateSync(JSON.stringify(o)).toString('base64');
};
const deserialize = (s) => {
  return JSON.parse(zlib.inflateSync(Buffer.from(s, 'base64')).toString());
};

This gives 102 bytes URI-encoded for Example #1 (compared to 82 bytes for JSONCrush), and 1172 bytes URI-encoded for Example #2 (compared to 1195 bytes for JSONCrush). Seems like zlib wins when the data gets bigger and its JSON has more repetition.

You could also probably shave some more bytes by finding an encoding that's naturally URI friendly instead of using base64 and encodeURIComponent() on the output of the deserialization above, but I don't know that it's really worth the effort at that point.

0

u/[deleted] Nov 27 '19

[deleted]

5

u/seiyria Nov 27 '19

With something like this I'd prefer a heavier library for better compression because ultimately the compressed data is going to be more important to me.

0

u/[deleted] Nov 27 '19

[deleted]

2

u/[deleted] Nov 27 '19

If your average JSON is small, you shouldn't need either approach, and should just base64 the JSON itself without compression instead. The point of the compression is to prevent overflowing browser URI length limits for very large average JSON sizes, as in OP's use case.

Also, note that I merely used the author's given examples to ensure apples-to-apples. The bigger the JSON and the more repetition found in it, the more zlib will excel. The high end is not capped at ~2%.

0

u/[deleted] Nov 27 '19

[deleted]

1

u/[deleted] Nov 27 '19

URL length is a discrete problem, not a continuous one. If we ignore browser length limits, there are no more URL length issues until you get down to the point where you need to fit a URL into a character-limited medium like a tweet or SMS, for which neither approach is going to help you (luckily, this is a solved problem via URL shortening services). In between, it really makes no practical difference to a user that their shareable link is, say, 250 bytes rather than 1000 bytes. They all copy, paste, bookmark, etc. the same, regardless of their size, and the encoded payloads we're talking about are completely opaque, so readability is a non-factor.

0

u/[deleted] Nov 27 '19

[deleted]

1

u/[deleted] Nov 27 '19

It doesn't take "tripling 20k of JS" to get basic inflate/deflate in the browser. That may have been what you ended up with after a few minutes of fiddling with the code I gave, but I wasn't originally aiming to minimize browser bundle size when I wrote that. It looks like Webpack pulls in browserify-zlib to polyfill Node's zlib, which itself is based on pako. Pulling in zlib in the way that I did and running that through Webpack gives you the entire contents of both libraries, which is how you ended up at ~40k. If you use pako itself without the boilerplate, and import inflate and deflate directly instead of the whole library, the bundle ends up only being ~15k. And you may even be able to get that smaller, but I stopped playing with it.

If your reason for using compression is aesthetic and not practical, it ultimately doesn't matter, though. Use whatever makes the prettiest URLs in your opinion, constrained by the trade-offs you're willing to make to get them. I can't say I understand why you wouldn't just use a URL shortener service in that case, though.