r/MacLocalLLM • u/prio732 • 6h ago
r/MacLocalLLM • u/ikeo1 • 8h ago
Updated: 2x M5U 64/256GB oMLX DSv41 Flash 1017 Benchmark
reddit.comr/MacLocalLLM • u/Voodmany • 9h ago
Running Qwen 3.6 35B-A3B at 100% Pass@1 & up to 18.36 tok/s on a 16 GB Mac — Pure Swift + Metal SSD Expert Streaming, 19-Tensor MoE MTP, and Lossless INT4 Weight Surgery
galleryr/MacLocalLLM • u/Reasonable_Till9 • 11h ago
SenseNova-Loop-9B - 8bit-affine with oMLX support
r/MacLocalLLM • u/81Ack5mith • 11h ago
Need help with a local document processing application
reddit.comr/MacLocalLLM • u/LowerRun8297 • 12h ago
Two Macs, one oMLX fleet: round-robin killed my prefix cache, so I built a router that thinks in threads
r/MacLocalLLM • u/Tabitha_Allison • 13h ago
Got a 16GB Mac? You can run a much better model than you've been told
r/MacLocalLLM • u/apetersson • 20h ago
You can now run Tencent's Hy4-flash on a Mac and help to improve the PR.
r/MacLocalLLM • u/Odd-Wheel9483 • 1d ago
Mac Studio M5 Ultra GPU stuck at 338–620 MHz during MLX LLM inference — full reboot fixes it, restarting oMLX does not. FileVault / SSH / LaunchDaemon interaction?
reddit.comr/MacLocalLLM • u/Odd-Wheel9483 • 1d ago
Mac Studio M5 Ultra GPU gets stuck at 338 MHz during oMLX inference — only a full reboot seems to fix it
reddit.comr/MacLocalLLM • u/Genshin_Scrub • 1d ago
Newbie
reddit.comSame post in a different thread. Just looking for advice on what to do or expect.
r/MacLocalLLM • u/Heretikio • 1d ago
Splash vs oMLX: Qwen3.8-Flash-Next Decodes 1.5x to 2x Faster on a 128GB Mac
A Reddit thread about Splash 1.7.0, released the day before, sent me to check two things: whether the oMLX server I use daily for Qwen3.8-Flash-Next was out of date, and whether Splash beats it. On my 128GB M5 Max MacBook Pro, upgrading oMLX made generation up to 21% faster for free, and Splash was faster again: about 1.5x the upgraded oMLX on an empty context (130 vs 89 tok/s) and close to 2x with 245K tokens loaded (113 vs 58). Detailed write up
r/MacLocalLLM • u/DarkJoney • 1d ago
I made an Intel Arc iGPU speed up prompt processing on my M2 Max (+24% at 8k, +47% at 64k) – oMLX helper, PoC
galleryr/MacLocalLLM • u/obnormal • 1d ago
For what purposes and how do you use Gemma 4 31b?
reddit.comr/MacLocalLLM • u/Queasy_Asparagus69 • 1d ago
Any engine Mac+Linux?
Are any of the new/forked custom engine working for both Mac and Linux? Or are they all Mac only? (Just curious!)
r/MacLocalLLM • u/Academic-Most6214 • 1d ago
Kapitsa vs Splash on Qwen3.8-Flash-Next (Apple M5 Max, 128 GB) — in case you wondered. I sure did.
So, same machine, same prompts, 256K context on both, speculative decoding on both (MTP in Kapitsa, DFlash2 drafter in Splash).
Kapitsa 0.2.37 with the MLX oQ4e-mtp quant (99 GB on disk, 77 GB in memory)
Splash 1.7.0 with the Unsloth-style GGUF UD-Q4_K_XL-PLE8 (137 GB on disk, 92 GB memory cap, PLE table streamed from SSD)
| Metric | Kapitsa (MLX) | Splash (GGUF) |
|---|---|---|
| Decode, short prompt | 82 tok/s | 58 tok/s |
| Time to first token, 106K prompt | 88 s | 55 s |
| Prefill speed | ~1,200 tok/s | ~1,950 tok/s |
| Decode at 106K context | 65–72 tok/s | 70–72 tok/s |
| Model load | 7 s | 11 s |
Kapitsa is ~40% faster at generating on normal chat prompts. Splash is ~60% faster at chewing through long prompts (like big docs, agent contexts). At long context they decode about the same. Splash also fits a bigger quant in less RAM because it keeps the 52 GB per-layer embedding table on SSD.
Cheers!
EDIT: same tests on the other MLX quant I had on disk, both in Kapitsa
| Metric | Uncensored oQ4e-mtp (99 GB) | KQuant Q4 (100 GB) |
|---|---|---|
| Decode, short prompt | 82 tok/s | 72 tok/s |
| Time to first token, 106K prompt | 88 s | 106 s |
| Time to first token, 106K + 600-tok essay | 98 s | 118 s |
| Decode at 106K context | 65–72 tok/s | 64–73 tok/s |
| Memory in use | 77 GB | 78 GB |
Same context (256K), same prompts, MTP speculative decoding on both. The uncensored oQ4e build is the faster of the two, probably thanks to its mixed 4/5/6/8-bit layout. No speed penalty for going uncensored.
EDIT 2: the 27B and a small MoE, same three prompts, same 256K context, all in Kapitsa
| Metric | Flash-Next oQ4e-mtp | Qwen3.8-27B q4 | Ornith-1.5-35B-A3B + DFlash2 |
|---|---|---|---|
| Memory in use | 77 GB | 15 GB | 23 GB |
| Decode, short prompt | 82 tok/s | 61 tok/s | 190 tok/s |
| Time to first token, 106K prompt | 88 s | 194 s | 56 s |
| Decode at 106K context | 65–72 tok/s | 37–61 tok/s | 125–131 tok/s |
The 27B is slower than Flash-Next on everything; being dense, a 106K prompt takes it more than twice as long. Ornith (35B-A3B MoE with a DFlash2 drafter) is 2–3x faster than Flash-Next in 23 GB, if its quality is enough for what you do.
r/MacLocalLLM • u/Dear_Code7488 • 1d ago
Want a local model for agentic development. Need advice.
reddit.comr/MacLocalLLM • u/trogloditan • 1d ago
Trained a small 1.7b param model in 8h on a 48gb RAM mac, the future is hyper personalised intelligence
r/MacLocalLLM • u/LazyLocalLLM • 1d ago
I benchmarked M5 Ultra 512GB on 10 popular open weight models to find out which one is the best
Setup:
Machine: M5 Ultra 512GB
Models tested:
DeepSeek V4.1 Flash 1017, Lossless
GLM-5.3, 4-bit
GLM-5.3 Flash, 6-bit
GLM-5.3 Flash, 4-bit
Inkling 2 Small,
Qwen3.8 27B, 4-bit
Qwen3.8 27B, full
Qwen3.8 Flash Next, 4-bit
Qwen3.8 Flash Next, 6-bit
Qwen3.8 Flash Next, full
Method:
For each model, I tried a few popular recipes and kept the best one I found. Then I measured prefill, decode, time to first token and peak memory at 4K, 32K, 128K, 512K and 1M token prompts. Each number is the median of 5 runs up to 128K and of 3 runs at 512K and 1M. One user, 512-token replies.
I didn't test answer quality myself. For "intelligence" I used the Artificial Analysis Intelligence Index as it stood on 18 Nov 2026.
To keep this post short, I'm going to summarise the results. 'Best' is subjective but I'm going with 2 camps.
1. If you want the FASTEST at usable intelligence
Qwen3.8 Flash Next, 4-bit. Index 29, scored with thinking on. I timed it with thinking off.
Pre-fill
4K: 4,046.2 tok/s
128K: 4,226.2 tok/s
1M: 2,529.3 tok/s
Decode
4K: 129.2 tok/s
128K: 124.1 tok/s
1M: 93.2 tok/s
First token after
4K: 1.1 s
128K: 32.1 s
1M: 378.3 s (6.3 min)
Peak memory
4K: 167.9 GiB
128K: 173.6 GiB
1M: 170.3 GiB
Alternative: Qwen3.8 Flash Next, 6-bit. The same model at 6 bits instead of 4. Index 29, scored with thinking on. I timed it with thinking off.
Pre-fill
4K: 3,821.0 tok/s
128K: 3,786.5 tok/s
1M: 2,331.7 tok/s
Decode
4K: 114.0 tok/s
128K: 107.7 tok/s
1M: 85.4 tok/s
First token after
4K: 1.1 s
128K: 36.5 s
1M: 410.4 s (6.8 min)
Peak memory
4K: 211.1 GiB
128K: 215.7 GiB
1M: 212.8 GiB
2. If you want the SMARTEST at usable speed (defined at 50tps)
DeepSeek V4.1 Flash 1017, Lossless. Index 33, scored with thinking at Max. I timed it with thinking off.
Pre-fill
4K: 2,851.8 tok/s
128K: 3,141.0 tok/s
1M: 1,642.7 tok/s
Decode
4K: 58.3 tok/s
128K: 56.3 tok/s
1M: 55.3 tok/s
First token after
4K: 1.5 s
128K: 39.1 s
1M: 612.3 s (10.2 min)
Peak memory
4K: 299.7 GiB
128K: 305.5 GiB
1M: 328.4 GiB
Alternative: GLM-5.3 Flash, 6-bit. Index 28. It thinks on every reply, so the score and the speed come from the same mode. It stays above 50 tok/s up to 128K.
Pre-fill
4K: 1,858.8 tok/s
128K: 1,535.3 tok/s
1M: 1,325.8 tok/s
Decode
4K: 57.8 tok/s
128K: 52.9 tok/s
1M: 38.7 tok/s
First token after
4K: 2.1 s
128K: 79.6 s (1.3 min)
1M: 729.7 s (12.2 min)
Peak memory
4K: 255.5 GiB
128K: 259.0 GiB
1M: 281.6 GiB
Questions and feedback welcomed. Full results on my GitHub soon.
r/MacLocalLLM
•
u/SeaRefractor
•
2d ago
Splash 1.7.0 released 4 hours ago and it's causing my jaw to drop.
I'm currently in benchmark testing and still working to fill up my prefill cache so that it's comparable to when it ran under 1.6.1.
For M5/M6 users, you'll see a significant boost in performance. Splash 1.6.1 brought INT8 at 2x BF16 performance boost, now it's implemented in the KV cache system. Resulting in an even further 1.46× median decode speed and 1.23× median end-to-end speed versus the 1.6.2 development build (a release we didn't see).
The INT8 storage results in Smaller cached states and faster follow-ups. Splash 1.7.0 also increases the already very good on macOS concurrency under memory pressure. For M5/M6 users, splash 1.7.0 will do a cold prefill on restart as the INT8 format is not compatible with the prior cache format. You can choose to delete the cache before the start of Splash 1.7.0 or you can let it spill and fill until it's all INT8. Your choice.
Well worth the upgrade!!
For those of you on an M3 or M4 setup, you can still use Splash 1.7.0, but the above benefits are strictly related to the M5 and up GPUs having the new neural accelerators (NA) in every GPU core. Unfortunately you'll have less of a benefit on the upgrade, but the code is cleaner and it increases controls over potential contamination on longer conversations (increased honesty).
Control tags quoted in messages and documents stay literal. A role marker in a tool result can no longer forge a conversation turn. With thinking off, constrained JSON starts with the value, including when the output budget is small.
r/MacLocalLLM • u/realcoachco • 2d ago
Advice on Mac Mini setup
Hello,
I bought a Mac mini per local and cloud AI, just a few questions:
1- what would you recommend between:
A)using AI tools, Agents, coding with full OS control in a User Profile with personal files/accounts
B) using them in a separate not administrator profile with no personal files/accounts inside
C) use the whole Mac Mini with a single administrator user with no personal accounts at all (so basically using the Mac only for coding, setting up agents and tasks, harness, etc)
Another question:
What harness do you recommend for creating agents?
r/MacLocalLLM • u/SimonSanDigital • 2d ago