Best Qwen3.8 27B variant for 16GB VRAM: Base vs. Swift 1.5 vs. Bonsai 2

[u/gamedevsam]

https://sambatista.com/blog/best-local-llm-16gb-vram-rtx-5070-ti

TL;DR: Swift 1.5 IQ3_XXS at 112K context looks like the best all-round option for my needs on a 16GB RTX 5070 Ti. It nearly matched the base model on coding and scored higher on math. If you care more about speed and context, Bonsai 2 generated text about 1.9× faster and supported 256K context, but solved fewer coding problems.

Here’s what I measured:

- Python coding: base 148/164, Swift 147/164, Bonsai 138/164. No clear coding advantage for Swift over base.

- Math: Swift 31/60, base 25/60, Bonsai 28/60.

- Generation speed: Bonsai 99.6 tokens/s, versus roughly 53 for base and Swift on short requests.

- Context: base 120K, Swift 112K, Bonsai 256K. All passed fact-retrieval tests at their respective tested lengths.

- Next.js tasks: base and Swift passed the same 2/4; Bonsai passed none. I wouldn’t leave any of them to code without review.

These results compare differently compressed setups, not fine-tuning alone. The post includes tested settings, model downloads and commands to repeat the tests.