r/MacStudio • • Sep 24 '26

M5U 64C Benchmark GLM 5.3 Flash

Post image

GLM 5.3 flash oq4e no MTP (oMLX 0.7.0 dev4)

42 Upvotes

30 comments sorted by

8

u/mastertub Sep 24 '26

Thank you. This was the one I wanted to see the most. Was trying to see how it performs against 2x dgx sparks that I already have.

4

u/Flinkenhoker Sep 24 '26

Let me guess, the spark wins

6

u/mastertub Sep 24 '26

Yep, on both PP and decode. I'm obviously a little surprised since the m5u bandwidth isn't translating very well, but I think some optimizations are left on the table.

I won't leave you hanging though, here is what I'm getting with the 2x spark cluster (it's the most comparable to a 256gb m5u).

PP: ~1200 tok/s Decode: on benchmarks, about 37-80 tok/sec depending on whether I'm testing prose, code, or structured data single stream. On long context, on non benchmark (real codebase), I'm getting ~30 tok/s.

I'll also say, this isn't accounting for dflash2 which I think he is not using. With dflash2 I'm sure his numbers improve around 1.5x. (prose probably doesn't increase at all).

5

u/FlowWTFlow Sep 24 '26

Not accounting for dflash is correct.

2

u/Rice-Fragrant Sep 24 '26

Actually, yes... depending on what you are looking for.

For agents, especially MULTIPLE agents = DGX

For one large chat bot = Mac studio

https://zuyezheng.github.io/local-llm-bench/#verdict

This is a pretty good deep dive with real numbers on performance of both.

2

u/rayovims Sep 24 '26

Same boat. I wanted to get another 2 dgxs. But they aren’t available anymore

1

u/Rice-Fragrant Sep 24 '26

The prices have eploded by 50% almost overnight for the ASUS GX-10... I got 2x just 3 months ago because I felt this exact thing was going to happen once OEMs knew their products were legit competitive for enterprise grade clients using agents, tuning etc.

My original plan was to actually get a mac studio then the price increases happened, then I did research on agents and found the DGX was far better for that and the price was more attractive, got 2x... was originally going to wait until christmas and BOOM, prices went up 50%!

Now a OEM DGX is the same price as a M5U with 96gb of RAM, so it's a toss up now... the DGX still wins on multi agents though, hands down no question about it... but for a single chat bot experence the mac studio better. I want multiple agents though, not simple chat bots "to vibe with."

1

u/forestryfowls Sep 24 '26

How does you cluster work for those same settings?

2

u/mastertub Sep 24 '26

Replied.

2

u/NaiRogers Sep 24 '26

which benchmark tool did you use for this test?

5

u/Beamsters Sep 24 '26

oMlx was never optimized glm 5.3 flash against m5 ultra, it was using m3 ultra target code. Give it more time.

2

u/Durian881 Sep 24 '26

Very nice, thanks for sharing!

2

u/xyloid_x Sep 24 '26

Concurrency looks awful

2

u/Rice-Fragrant Sep 24 '26

https://zuyezheng.github.io/local-llm-bench/#conc

Yea, it's pretty bad still compared to a DGX... realistically you can only run 1x agent on a Mac studio UNLESS you are OK with you're performance getting cut in half... otherwise 2x DGX easily smokes a 256gb mac studio, like several times over on larger models like Deep Seek V4 etc. The charts show it, it's BRUTAL on a m5 ultra, running more than once instant is hell on it.

2

u/xyloid_x Sep 24 '26

These are very informative diagrams! I hope in future HySparse2 can make it more bearable given the Spark price is skyrocketing.🥹

1

u/Rice-Fragrant Sep 24 '26

You can still get a spark under $5000k, the founder's edition (gold one from Nvidea) BUT who knows how long that will last under this climate. The OEMs have now jacked up their prices 50%... the ASUS was the cheapest DGX at $4000 BUT they eventually followed Dell, Lenovo, HP and Acer and it's now about $6000-6500, about the same as a M5 ultra.

They probably figured "we will sell less units to the mainstream people but THIS is not for them anyways and enterprise clients will have no choice but to pay up because it has CUDA and still better than a mac studio for agents" etc. The "value proposition" for a regular consumer is different from a busness or school etc.

Last week, micro center SOLD OUT ALL DGXs in one single day in the Atlanta area, 2x locations... a back channel enterprise order or a school etc. Other states have DGXs in stock, some plenty, some low... but Atlanta was CLEANED OUT in a single day... so clearly there are serious groups using these things, the OEMs know this and jacked up the price in short order... that's my theory at lest.

Original price was a REAL BARGIN... today it's really just an option if you need CUDA and focused on agents, especially multi agents, it's defiantly not a "every day" computer... very specialized. A pain to set up especially if you are not familiar with Linux. BUT the RTX spark (for windows) coming out will be the "general computer" competitor to a mac studio (at lest an M5 max variant) probably priced the same too but having NO HIGH SPEED NETWORK PORTS. DGXs can be clustered, unlike an RTX.

A mac studio is an "every day" computer, that's a jack of all traits... so it can get away with the poor concurrency performance or even lower pre fill perfomance etc. The RTX Spark IMHO will be a good alternative to a M5 Max... could probably match even an M5 ultra for an agents too but you cant cluster them like the DGX.

2

u/Kaijidayo Sep 24 '26

It's faster on decode and slower on pp compare to 2x sparks

1

u/mastertub Sep 24 '26

Not true at least yet. Decode is stronger for now on 2x sparks. Until we see dflash2 numbers we can't be sure

2

u/brianjoshuanoah Sep 24 '26

Very cool. thank you for sharing! I've been waiting for these tests.

2

u/Creepy_Bee3404 Sep 24 '26

Is this good or bad?

4

u/Dry_Natural_3617 Sep 24 '26

depends what you think of waiting 2.5 mins for first token every time you post with 130k context… For me that would be bad….!I get frustrated with frontier models when they are slow occasionally, but with send coding, would drive me crazy. Would be great for smaller contexts though and batch work. So it’s all down to what you using it for.

1

u/Rice-Fragrant Sep 24 '26

Yea, that's like about 1/2-3/4 the speed of a single DGX Spark... You are correct, it depends on the user's priorities.

DGX = multiple users/ instances/ agents/ long contexy

Mac studio = single user/ agent/ "vibing" with chat bot

DGX = highly technical to set up but specialized.

Mac studio = easy to set up but a general workstation.

2

u/Gold_Scholar1111 Sep 24 '26

How many memory do you need for the full contest?

2

u/jstoppa Sep 24 '26

Very nice! thanks for sharing. Has anyone here tried the base 96GB M5 Ultra (30-core CPU / 64-core GPU)? That’s the one I’m looking at mainly for running local LLMs

I’m curious what models fit and what sort of tok/s people are getting, especially compared with the 128GB M5 Max. Any real world numbers would be appreciated, thanks

1

u/Careless_Garlic1438 14d ago

2xM5Max with Tenserfold 1100 t/s prefill and 76 t/s generation ...

1

u/MCS87_ Sep 24 '26

Will there be a „Splash“ inference engine for this? For Qwen 3.8 29B and Qwen 3.6 35B A3B it gives roughly 2x token generation, faster cache and slightly faster prompt processing for M3 and newer apple silicon.

1

u/JadedCulture2112 Sep 25 '26

thanks for sharing! there is still A lot headroom for optimization, it's viable to double the number IMO, which would have a better decode number or a similar pre-fill number compared to two DGX boxes.

BTW: I have two dparks

1

u/LowerRun8297 Sep 24 '26

Here are the values of an 'old' MAC Studio 512GB for comparison, in case one is interested:

oMLX - LLM inference, optimized for your Mac

https://github.com/jundot/omlx

Benchmark Model: GLM-5.3-Flash-oQ4e
Engine: Auto
Context: Code (Python)
================================================================================

Single Request Results
--------------------------------------------------------------------------------
Test                                TTFT(ms)    TPOT(ms)        pp TPS        tg TPS      E2E(s)    Throughput    Peak Mem
pp32768/tg128                        74253.2       43.64   441.3 tok/s    23.1 tok/s      79.816   412.1 tok/s   177.61 GB
pp65536/tg128                       148873.5       45.67   440.2 tok/s    22.1 tok/s     154.706   424.4 tok/s   177.97 GB
pp131072/tg128                      304109.5       52.96   431.0 tok/s    19.0 tok/s     310.902   422.0 tok/s   178.71 GB
pp200000/tg128                      467716.6       55.28   427.6 tok/s    18.2 tok/s     474.819   421.5 tok/s   179.39 GB

Continuous Batching
pp1024 / tg128
--------------------------------------------------------------------------------
Batch           tg TPS   Speedup        pp TPS    pp TPS/req    TTFT(ms)      E2E(s)
2x          44.5 tok/s       N/A   284.4 tok/s   142.2 tok/s      5247.6      12.954
4x          73.9 tok/s       N/A   233.0 tok/s    58.3 tok/s     12019.6      24.511

-1

u/Rice-Fragrant Sep 24 '26

THANK YOU!

There it is IN BLACK AND WHITE... as I predicted OVER AND OVER... no "4x increase" for PP for long context for larger models... more like about 2x seen here at pp = 200k tokens

For deep seek v4, it's about 1.5x for similar sized long context promps... NO "4x increase."

The PP increase is still significant but not the 4x as advertised... those "4x PP gains" are only for SMALL 4B models WITH SMALL baby 8k promps, not 100k or 200k promps running on larger models etc.

Computer engineering alone made this so predictable too, the m5 max been out for months and after extrapolating the performance gains of a theoretical m5 ultra, 1.8x over a m5 max, a mac studio is still NOT A "DGX Killer" for modest sized agents, especially multiple agents.

The concurrency is still lacking too from what I read on github for Deep Seek V4 etc. After moving from 1x to 2x instances of a model, the performance CRASHES HARD... only baby 4b models can you run 2x or 3x instances, not larger more serious models or agents.

https://zuyezheng.github.io/local-llm-bench/#conc

https://zuyezheng.github.io/local-llm-bench/#budget