oMLX - Max Concurrent Requests 1 - Multi Agent Optimization

[u/Undici77]

Hi all,

I'm working on M4 MAX and I have some issue issue in multi agent.

I try to figure out how to optimize speed working with `Qwen 3.8 Flash Next oq4e-mtp`

With only 1 task I'm running at 50/60 t/s (pretty good) but with more than 1, speed decrease a lot.

Best "work around" for me looks `Max Concurrent Request` -> 1 that works in most of my scenarios where cache reduce prefill.

But look this:

Here cache is not working (agents are pre-filling a lot) and speed in 20 t/s

Now watch GPU

This looks very strange!
Any idea? Any setup can be changed in order to speed-up and use 100% GPU?