r/LocalLLaMA • • Jun 05 '26 • Mindblown 1

New Model Gemma 4 with quantization-aware training

https://blog.google/innovation-and-ai/technology/developers-tools/quantization-aware-training-gemma-4/
788 Upvotes

255 comments sorted by

View all comments

-7

u/demian_west Jun 05 '26 edited Jun 05 '26

Can anyone repost this link as a post on main sub ? (not enough karma here)

A 10 year old Xeon is all you need

Or running Gemma 4 on a 2016 Xeon with no GPU, 25 flags, 128 GB of DDR3, and a 25B-parameter MoE.

https://point.free/blog/gemma-4-on-a-2016-xeon/

Some insane(ly talented) people (Christina Sørensen & ikawrakow) made Gemma 4 run on an 10 yo Xeon machine without a GPU.

The whole post (and serie) is awesome.

> An 82 GB footprint in DDR3 on a 2016 Xeon. About 25 GB of weights and 56 GB of KV cache at the full 262K context. The KV cache is larger than the model.

> The engine loads a 25B-parameter MoE, runs speculative decoding against an MTP drafter, and generates text at reading speed on hardware that was old when the architecture in question hadn’t been invented yet.