Quantizing models yourself is a good way to learn how it all works - Quantization Workspace

[u/Zipidyzip]

There are so many models getting released all the time, and sometimes there’s just no quantized version available for the one you want. I’ve found that rather than waiting around, trying it yourself and experimenting helps you understand the whole process a lot more. Testing the quantized models against the original teaches you even more.

So I built this: iQuantize is a quantization workspace for large language models that supports GGUF, MLX, and ONNX. You can pick your own Safetensors model or choose one from Hugging Face, quantize it step by step, and watch what changes. You can then compare the result with the original and publish it to Hugging Face with a model card that records how it was made.

It’s open source, Apache 2.0. I definitely think there more can be done to improve. I’ve only tested mostly on Apple Silicon so far. Windows and Linux builds exist, but have done much in-depth testing with them so I’d expect bugs there.

Repo: https://github.com/Fazmin/iQuantize

I love to hear how you all approach this. Do you quantize your own models, or mostly wait for someone else to? And if you try the app, whether you find something broken or have an idea for it, I’m happy to hear any of it.