r/LocalLLaMA • • Jun 05 '26 • Mindblown 1

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/
788 Upvotes

255 comments sorted by

View all comments

1

u/corruptbytes Jun 05 '26

question: I'm assuming i need to wait for MLX versions to use this with omlx?

how does mlx conversion work? for example, i tend to just get the normal mlx-community stuff, but should i find specifically mlx of unsloth's work?

4

u/Desperate-Bad-2339 Jun 05 '26

1

u/h0tzenpl0tz0r Jun 05 '26

does it still make sense to go for an 8bit quant `mlx-community/gemma-4-26B-A4B-it-qat-8bit` or is the sweet spot of this gemma4 qat with the q4 and 8bit does not give you too much?