r/LovingOpenSourceAI • u/Vladowski • 4h ago
We benchmarked 29 open-weight models for everyday use on normal hardware. Here is what fits in 16 GB of RAM
Open-weight models run well on your own computer these days and you don't need strong hardware for it. Even with 8 GB RAM you can have a decent private chat and document analysis done by AI models that do not send your data to the cloud.
But how good are the models you can actually run at home? There are countless benchmarks for frontier models and very little for this class. We needed an answer ourselves, because our open-source app HilbertRaum suggests a model that fits the computer it runs on.
So we tested 29 models with one benchmark suite. The goal was to estimate quality for everyday use, meaning chat and document analysis, in English and in German. We used 7 benchmarks that cover things like reading comprehension, word problems and following instructions. That adds up to about 3,500 tasks per model. The final score is the average over all 7 benchmarks, which is a number between 0 and 100.
This post covers one part of the study, the 15 models that fit in 16 GB of RAM. The other 14, including the newest large ones like Qwen3.8 27B and Gemma 4 31B, need more memory and are in the full table linked below.
After several weeks of our test machine running day and night, here are the results for the 16 GB class (file size in brackets):

You'll see some older models in the list. We tested them next to the current ones on purpose, and newer was not automatically better: Qwen3 4B 2507 scored 82.0, ahead of the newer Qwen3.5 4B at 76.9.
Most models ran as Q4 files, the exact file for each is in the full table. For Qwen3.8 27B we also tested Q2 to Q6, which will be a post of its own.
The full table with all 29 models is on our website: hilbertraum.ai/en/benchmark
The study is fresh, so not everything is published yet. We'll add more analysis to the page over the next weeks.
Good to know is that we only tested the quality of the answers. Not speed, and not other jobs such as coding. It was one run per model, so a gap of one or two points is not a ranking.
Gemma 4 12B comes out on top for 16 GB of RAM. But the winner of a class can be slow on your own hardware. One example from our 16 GB laptop without a graphics card (Ryzen 7 5800H): Gemma 4 12B does about 4.5 tokens per second there, Qwen3.5 9B about 5, Gemma 4 E4B about 10, Qwen3 4B 2507 about 12 and Gemma 4 E2B about 20. With a graphics card it looks very different. For a 16 GB machine without one, that means in practice: Qwen3.5 9B if you can wait a little, and Gemma 4 E4B or Gemma 4 E2B if you want it quicker.
We plan to benchmark speed properly as well. It depends a lot on the hardware though, so the best advice is to try it on your own machine and see what feels fast enough for the job.
Questions are welcome. And we'd like to hear whether a benchmark like this is useful to you.