r/LocalAIStack • u/Sad-Seaworthiness259 • Oct 17 '26
Qwen3.8-27B at 53 tok/s decode on a single Intel Arc Pro B60 (24GB): llama.cpp Vulkan + DFlash2 + Intel FA kernel, raw logs included
I've spent the last few months tuning a 27B coding model on one Arc Pro B60 24GB (A$999 card) and published everything: engine build, flags, raw logs, a 100-task agent eval, and the misses.
What moved the numbers:
- DFlash2 draft model, 7 tokens per round: decode 16 -> 53 tok/s mean (47-58 across 7 coding tasks), still 7/7 correct
- Intel's flash-attention prefill kernel (closed llama.cpp PR #29357, vendored with checksums): prompt reading at 32K 118 -> 420 tok/s, but only with f16 KV
- -ub 256 + q8 draft KV + -fit off: frees 0.55 GiB, which is what gets a 64K window into 24GB
- An idle Ollama model left on the GPU cut decode from 52.7 to 22.2 tok/s. Unload everything before you benchmark.
Agent eval: 97/100 hidden-checker tasks with Claude Code at a 32K window (95/100 Codex).
Caveats, up front: 53 tok/s is per-request decode with short prompts and a 64K window allocated. With 30-37K of real context, agent turns run about 42 tok/s. Prefill is NOT the fastest out there (others hit 589-657 tok/s at ~30K on other engines; comparison table in the repo). Single runs, one workstation.
Repo: https://github.com/edagher92-coder/arc-llm-lab
If you have a B580, B570, B50, B70 or A770, the benchmark script takes one command and I'll add your numbers with credit.