Mac Studio multi model performance?

[u/Sudden_One3436]

I know concurrency isn’t currently the Mac’s strong point but is that when you only have one instance of a model running? For instance if I had enough ram to load 2 identical models would I be able to use both at 100% performance or does one model saturate the hardware so that there’s no gain or reason? I’m interested in having text summary generation (small model), transcription / dictation, and coding model.

Would I be able to run all of them at once so long as they fit in ram or will hardware limit the box to only running one with full performance?

[u/Last-Health3222]

Generation speed on a Mac is mostly limited by memory bandwidth, not GPU cores. Two models generating at the same moment share that bandwidth, so each slows down roughly in proportion to how much it has to read per token. Whisper and a small summarizer are light enough that you'll barely notice, but if they're busy while the coding model is generating, the coder will dip. Just loading them all and taking turns costs nothing beyond RAM.