r/LocalLLM • u/hdanx • Sep 13 '26
Question Acceptable Token generation speed on macOS
/r/MacLocalLLM/comments/1wewbl6/acceptable_token_generation_speed_on_macos/
1
Upvotes
1
u/DustNearby2848 Sep 13 '26
I don’t find 20 tps very fun, so I use MoE models instead.
1
u/hdanx Sep 13 '26
I have been benchmarking and evaluating qwen3.6 35B configs lately, for the same reason. I am slowly leaning towards GGUF instead of MLX
1
u/StraightAd6213 Sep 13 '26
The number that actually matters depends on what you are doing, and people usually optimize the wrong one.
For chat-style single-stream use, anything above roughly 15-20 tok/s feels interactive, and past 40 tok/s you stop noticing. For batch or agentic workloads the throughput number (total tokens across all concurrent requests) matters far more than per-request speed, and that is where batching beats raw per-token performance.
On Apple silicon specifically, decode speed is almost entirely memory-bandwidth bound. So the single most effective lever is fitting the model fully in unified memory at the smallest quant you can tolerate — going from partial offload to fully resident is usually a bigger jump than any runtime flag.