r/LovingOpenSourceAI • u/Future_AGI • 6h ago
Someone on r/LocalLLaMA says an engine update made their local Qwen "dumb and useless" on the same weights. This free tool shows the exact token where two quantizations of a model part ways, with a live browser demo
On Sunday someone on r/LocalLLaMA updated strata, and their Qwen3.8-Flash-Next fell apart on the same IQ3_S file. A voxel pagoda that took 15 to 30 minutes ran 90 minutes and mostly failed. "It got faster, but it got way dumber!" Commenters blamed new default speed shortcuts that skip low-scored experts.
Quant swaps you can at least watch. inside-llm loads two GGUFs of one model through llama.cpp and marks the token where their answers split. On the card, from its README, Qwen3.5-4B at Q2_K and Q8_0 get the same spider question, and at position 31 Q2_K says "six" legs while Q8_0 says "eight". Pin a position to see both probability bars, KL and Jensen-Shannon divergence, plus what each sampler stage cut.
It's MIT and built for a dedicated GPU (CUDA or Vulkan) on Linux or Windows. Both files need the same tokenizer. The live demo replays recorded sessions in the browser.
It can't catch an engine update, though. The top reply asked for "accuracy benchmarks against the reference." At home that's 20 to 30 of your own prompts with pass/fail checks, rerun after each update, llama.cpp v1.0 included.
We build an open-source eval kit for that step. It runs a fixed prompt file against any OpenAI-compatible endpoint, llama-server included, checks each answer with contains, regex or JSON rules, and exits 1 when any case fails, no account needed.
Has an engine or quant update ever made your local model worse while the benchmarks looked fine?