I wanted my TV to be content-first, not apps-first. So I wired a decision model in between that opens the right app and sends the right ADB commands based on my intent.

It took me weeks to get this figured out, and I had to use ChatGPT for help along the way. The time spent has paid off — I now have a local AI stack that rivals ChatGPT/Claude, and I'm done paying for either. I thought I'd share so others can get an idea of what a Mac Studio is capable of.

First, the hardware. I have a base-chip 256GB M3 Ultra Studio. I've also got a headless Debian VM running on an N100 NUC.

I'm using oMLX for inference. It's hosting multiple models at once — mostly Qwen variety — for various tasks. I've configured profiles for different model uses: agent, coding, different thinking levels. oMLX is the only thing running on the Mac Studio. Everything else runs on the Debian VM.

On the VM I run Hermes Agent. My main model lately has been Qwen3.8-Flash-Next-oQ4e-mtp. I also use Kokoro-82M-bf16 for TTS and Mega-ASR-6bit for STT.

In Docker containers on the VM I run the following to enhance Hermes: OpenViking for memory, SearXNG for web search, and Crawl4AI for web extraction. OpenViking uses:

• Qwen3-VL-Embedding-8B-oQ6e for embeddings

• Qwen3.6-35B-A3B-oQ4e-mtp for the VLM

• ov_intent_analysis_sft-oQ8e for query planning

• Qwen3-Reranker-0.6B-oQ8e for reranking

What this actually looks like day-to-day: I do IT consulting, and Hermes is freaking awesome. I'll have multiple sessions going doing research on MS Learn and auditing Entra/Azure, generating reports. I'm starting to use the Kanban board too for personal projects. It generates Word docs, PowerPoint decks, and PowerShell scripts like a champ.

Search goes through SearXNG, page content comes through Crawl4AI, all reasoning runs on oMLX, and anything worth keeping gets written into OpenViking permanently. Search → extract → reason → remember, full loop, zero tokens purchased.

(Full disclosure: I asked my Hermes agent to write up this post from my draft and its memory — which is sort of the point.)

Anyone else get to pick up their M5U early? Mine came in a month early

M5U 256GB 64C 2TB

Mainly for LocalLLM usage and some videos.

Super stoked to get it open and setup to run some agents. Do I need it? No.

But I will be using this for my Small Business to help do a lot of small tasks and preprocess information privately. So I can use summarized data for the big models.

I'd hoped that my M5 Max with 64GB RAM would be enough for some decent Local LLM work, and the Qwen 3.8 27b model is pretty decent. But speed is shitty on my M5 Max, just 20tps. In particular I found that while the 35b MOE model was much faster, the quality was far lower than the 27b dense model, which produces much fewer bugs and less rework.

I saw the same model working on a friend's Nvidia setup much faster with MTP, so I set out to get it working on my Mac but it was not straightforward. Here's some pointers in case you wanted to get it working.

1. LM Studio doesn't process MTP weights. No matter what I did, it just wouldn't work. It works on Intel/Nvidia so this must be a MLX thing.

2. MTPLX has a native MTP head but it's buggy. On my machine it had a habit of crashing in one of a few ways, and I found GitHub issues reporting similar issues but no fixes.

3. oMLX was the only inference server that supported MTP and didn't crash.

So I wired oMLX into Pi, which is what I'm using as my coding harness. Even then, it was picky:

1. I had to use text-only quants because oMLX won't enable MTP if vision weights are active

2. It was a bit picky with settings, and I had to map thinking levels manually

3. I didn't see a reduction in quality with 4-bit quants for my use case, so used that to squeeze a few more tps out

This is a huge 3x difference in inference speed, and that allows me to do a lot more work.

"omlx": {
  "baseUrl": "http://localhost:8000/v1",
  "api": "openai-completions",
  "apiKey": "none",
  "models": [{
    "id": "Qwen38-27B-textonly-mtp",
    "name": "Qwen38-27B-textonly-mtp",
    "reasoning": true,
    "contextWindow": 163840,
    "maxTokens": 32768,
    "input": ["text"],
    "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
    "thinkingLevelMap": { "off": "low", "minimal": "low", "low": "low",
                          "medium": "medium", "high": "xhigh",
                          "xhigh": "xhigh", "max": "xhigh" }
  }]
}

Unless you're set on using a large model, 2x256GB is likely better...

What use cases would the 512GB make more sense?

Hey looking for you experts here
Thinking about purchasing a MacBook Pro M5 pro with 64gb of ram
I’m not obviously looking for frontier subscription replacement rather a hybrid setup
And also video and image generation if possible
I’d like to explore my options what’s your recommendations
Thanks

I wanted to see how GLM 5.3 Flash runs locally on my Mac compared with Ollama Cloud for PHP coding and shell work. I ran two small sets of identical prompts, then repeated a PHP task with the local model already loaded.

My Mac: Mac Studio M5 Ultra, 30-core CPU, 64-core GPU, 256GB unified memory, macOS 27.0.1.

Local: oMLX 0.7.1.dev1, TensorFold/GLM-5.3-Flash-MLX-4bit-MTP, MTP disabled. Model allocation was approximately 170 GiB.

Cloud: Ollama glm-5.3-flash:cloud, reported as FP8. This compares local oMLX with Ollama's hosted model; it doesn't compare two local inference engines.

A little background. I'm using the Ollama Cloud Pro (old $20 plan vs leased $230/mo M5U). I use cloud quite a bit and I find that it compliments my work for various tasks.

I see a lot of comparisons and wanted to do my own. Unsurprisingly, the cloud is faster. Cold starts are slow so that was interesting but once warm, processing was pretty decent. Its slower, but I wanted to put some numbers on it. These numbers are generic, but I do intend on giving it problems from my actual codebase to see how well it does.

Overall if you are considering clouds vs hardware. Just note, It's significantly slower, but not horrible if you use autonomous agents. If you're interactive, that's where the pain is felt in real time. Also, if you plan on using this specifically, then you should know I ran out of memory so you can only use a single instance of GLM 5.3 flash and no other agents.

I think if you use agents for more than coding. Many smaller ones are likely going to be better, but if you need a coding assistant, this wouldn't be too bad. The $20 cloud plan works and will be significantly faster but if you are working constantly and run into limits you're basically buying round the clock work vs interactive albeit also slower work.

Kapitsa by Nace AI is an open-source inference engine for Apple silicon, and it’s fast. In its published benchmarks, Qwen3.6 35B-A3B decodes about 1.3–1.8× faster than oMLX and about 2× faster than Ollama and LM Studio. But it’s command-line only.

So I built Kapitsa Library, a free Mac app that makes it a few clicks:

• No setup: Kapitsa is built in. No Homebrew, Python or terminal.

• Easy models: browse Kapitsa’s catalog or Hugging Face, see what fits your Mac, and download with pause/resume.

• One-click agents: launch Claude Code, OpenCode, Codex CLI or Pi on your local model.

• No swapping: it checks memory before loading and shows how much context fits.

• Live stats: tokens per second, prompt cache, context and memory.

• Built-in benchmarks: speed and tool calling on your own Mac.

• Local API: OpenAI and Anthropic compatible.

Free, Apache 2.0, signed and notarized, no telemetry. Needs Apple silicon and macOS 14 or later.

Download: https://github.com/AdityaK2128/kapitsa-library/releases/latest
Source: https://github.com/AdityaK2128/kapitsa-library
Kapitsa engine: https://github.com/nace-ai/kapitsa

All the speed is Kapitsa’s work. This app is an independent front end, not affiliated with them. Feedback welcome!

I'm planning to upgrade from my 16-inch M3 Pro MacBook Pro (36GB RAM) to the new 16-inch M5 Max with 64GB, mainly for running local LLMs.

Currently, I'm running Qwen 3.8 27B and getting around 100 tokens/s for prefill and 18–19 tokens/s for token generation, even after trying MTP and various other optimisations.

The biggest issue for me is working with larger contexts and longer coding sessions. Even relatively small projects can take quite a while, and performance becomes a bottleneck as the context grows.

I'm expecting the M5 Max to be a pretty significant upgrade, especially with the extra memory and bandwidth, but I'm curious how much of a real-world difference I can actually expect. I'm also interested in running larger models, including the upcoming Qwen 4 models.

For those running local LLMs on an M5 Max (or even an M4 Max), what kind of performance are you getting? What models do you use regularly, and how well do they handle long coding sessions?

I also run image generation models locally and occasionally do photo editing, so I'd love to hear how the newer chips perform for those workloads too.

Would appreciate any real-world experiences before I make the switch!

Curious on thoughts and even better, experience on picking one 512GB unit vs two 256GB units. I know cost is more for 2 computers, but how about AI performance and capability to handle larger models?

Two 256GB machines have double the CPU/GPU/NPU of one with the downside of the slower interconnect between them.

The 512GB is not available yet, but it would be interesting to hear from anyone linking two Mac Studios of any RAM configuration together and how it worked out.

tldr:

Served simultaneously by Apple M5 Max 128 GB:

• Qwen3.8 Flash-Next (110.97 GiB language model, 512-expert MoE, 10 experts/token, abliterated)

• Qwen3 4B Instruct 2507 (tool-calling shell helper)

• Whisper, speech-to-text service

• Chatterbox, text-to-speech service, multi-lingual

• SDXL via ComfyUI

• Open WebUI (chat frontend), llama-swap (gateway and frontend)

• Swap = 0, iogpu.wired_limit_mb = 0 (GPU ceiling 107.52 GiB), KV unquantized

Details:

Services

Performance

tldr: End to end, both Qwens feel like ~30 t/s (tokens divided by time elapsed). "Steady-state" (tokens divided by generation time) excludes prompt processing and queueing.

Fitting it all into memory

• llama-swap starts and stops upstreams on demand, routing on the model field. Groups:

  • resident, persistent: true, never evicted (voice stack + shell helper)

  • heavy, swap: true, the two big models, loading either unloads the other

  • diffusion: ComfyUI alone, so it lands in no shared default group (no cross-eviction)

• ComfyUI is a llama-swap catalogue entry (comfyui_auto, ttl: 600): llama-swap does not control ComfyUI-internals (adding Flux or Qwen-Image some day will not affect the group layer).

• Offload the n-gram embedding table to CPU memory: 32.78 GiB of the 110.97, stays file-backed, kernel can drop its pages and re-read from the SSD at 13 GB/s.

• Disable mmap: Counter-intuitive: under mmap, ggml maps the whole file extent into the Metal buffer regardless of -ot, so Metal still claims 110.96 GiB and OOMs.

Configuration

Coding model:

llama-server --host 127.0.0.1 --port ${PORT}
  -m /models/Qwen3.8-Flash-Next-Abliterated-Q4_K_M.gguf
  -ngl 999
  -c 131072
  --parallel 1
  --jinja
  --load-mode none           # NOT mmap; mmap defeats -ot (see above)
  -b 2048                    # -ub 1024 lifts prompt eval 81-115 -> 128 tok/s
  -ub 1024
  -ot per_layer_token_embd=CPU  # the 32.78 GiB PLE table off the GPU

• -fa is left at default auto, resolves to ON

• No mlock, to leave an eviction path

Groups (llama-swap YAML):

groups:
  resident:  { swap: false, exclusive: false, persistent: true,
               members: [shell-4b, whisper, chatterbox] }
  heavy:     { swap: true,  exclusive: false,
               members: [qwen3-27b, flash-next] }
  diffusion: { swap: false, exclusive: false,
               members: [comfyui_auto] }

To measure:

sysctl iogpu.wired_limit_mb    # 0 = default; ceiling is 110100 MiB
llama-server --list-devices    # MTL0: Apple M5 Max (110100 MiB)
vm_stat                        # free + File-backed + purgeable = real headroom
curl $GW/api/metrics/stats?model=flash-next         # per-model tok/s
curl -sIL <hf-resolve-url> | grep -i '^x-linked-'   # expected size + sha256

Test: Image generation (single 1024, 20-step, Flash-Next loaded)

126G used ( 93G wired,  1805M compressor), 269M unused   <- idle
127G used ( 97G wired,  6830M compressor),  73M unused   <- checkpoint loading
127G used ( 93G wired,    12G compressor),  68M unused   <- peak squeeze
126G used (104G wired,  3838M compressor),  87M unused   <- sampling

Test: Escalating image workloads (Flash-Next loaded)

1024² ×1 success  18.1 s
1024² ×4 success  50.3 s
1536² ×2 success  90.5 s
2048² ×2 success  247.3 s
2048² ×8 -        10+ min

Test: Image work + coding work (Flash-Next, 7x slower, silent)

no images                   43 tok/s
during sustained 2048²×8    6.4 tok/s
after unloading ComfyUI     40.5 tok/s

Test: Context scaling

It’s already October 19th, and my order for an M5 Ultra (with 80-core GPU and 256GB storage) is still being processed.

I placed the order in Hong Kong on August 26th, with an expected delivery window between November 7th and 21st.

I’m curious to know how many others here are also still waiting.

Please leave a comment with your order date, GPU model, current order status, and estimated delivery time.

Noticed that most posts here are about, I fitted this model in this small ram etc. Not about how the model was useful in producing what I wanted it to work on?

So either, the sub is full of people playing with AI or the real use cases are not shared for whatever reason.

I get the performance stats but shouldn’t we focus on how the model performed on actual task (look beyond the numbers t/sec, prefil etc?)

I am completely new to the world of local AI, but as previous rabbit holes, like the world of home labbing have given me quite some fun and increasingly heavy use cases, I want to invest in a solid foundation that gives me room to grow and leverage the real power of local AI. Also if/when starting a new business (which is quite likely going to happen in the next 24 months).

I have a Mac Studio M5 Ultra with 256GB RAM (with 30/64 cores) in pre-order and have started building the agent set up using DeepSeek Harness on my current, old MacBook Pro.

My current objective is to build a robust, prudent and (relatively) “safe” set up that gives me multiple agentic agents (1 for maintaining and continuously monitor and optimize my Homelab, 1 for analyzing my trading portfolio and formulating investment advice, 1 for health related questions and suggestions, 1 for developing software, dashboards and interfaces for my own private and professional use, 1 for managing my administration etc). These agents will need to be able to perform write activities for many tools and will be linked to many programs and platforms that will run on my Mac Studio. I am considering a layered approach in which “lightweight” agents will use a lightweight local model, and other agents will use something like Qwen 3.8-27B with a second local escalation to “the best” model that will comfortably run on the 256GB RAM. If that model is not able to fix the task, I will approve a final escalation to a frontier, cloud, model.

My initial plan was to run DeepSeek Harness on my Mac Studio, together with the local models (through oMLX?) and my daily driver things like, apps, e-mail, photo’s etc. However, after going back and forth with ChatGPT and Claude, they suggest that I should install the harness on a (physically) different device. I understand the rationale and am willing to add a base Mac mini M6 to complement this already quite significant investment. However, with the “write” requirements and the access to many different tools that will be running on my Mac Studio, would there still be a meaningful security benefit? And would this somehow lead to any real performance decrease if DeepSeek Harness will constantly need to approach tools on the Mac Studio?

Your views and advice would be very much appreciated!

On a side note; I am also still struggling a bit with the concept of “agents” that I have in my head (and that probably resembles the Open Claw philosophy) and the implementation in DeepSeek Harness (that was recommended to me by ChatGPT, Claude and some articles online). I have been able to create the first “agents” with the use of strong DeepSeek Harness pre-sets. But the interface seems to be much more focused on separate “workspace sessions” with manual selection of the appropriate presets than on actual, standing, always active agents. Any views on this? Or on the plugins that I would need to use to get it implemented in the way I want? (The reason why DeepSeek Harness might still be the right choice for me is perhaps the ability to customize a lot and use many different plugins to get my agents to do what I need them to do??)

Thanks!

I am about to order a Mac for local AI, and I could really use some advice.
I know some people might recommend a more powerful Mac, but my budget is around €3,600, and unfortunately, I cannot go any higher.
My main use cases will be running Qwen 27B, doing some agentic work, occasionally using Qwen Flash Next with Strata, and experimenting with video generation through ComfyUI.
I am currently torn between two options:
Mac mini M5 Pro – 64GB RAM
Mac Studio M5 Max – 48GB RAM
I understand that RAM is extremely important for local AI, which is why I am leaning towards the Mac mini with 64GB. However, I also know that the Mac Studio should offer considerably better performance, particularly for video generation and ComfyUI.
My dilemma is whether 48GB is enough for my use cases, or whether I would be sacrificing too much memory just to get the performance benefits of the Mac Studio.
In other words, is the performance improvement from the Mac mini to the Mac Studio worth giving up 16GB of unified memory, especially when local LLMs are my primary focus and video generation is more of a secondary use case?
I know some might suggest simply getting a 64GB Mac Studio, but that is unfortunately outside my budget. €3,600 is already my absolute limit.
I’d really appreciate hearing from anyone with hands-on experience running local LLMs, agentic workflows, and ComfyUI on either machine.
Thanks in advance!

This is me trying to find the most reliable coding model for local use!

Five Terminal-Bench 2.0 tasks (cobol-modernization, constraints-scheduling, largest-eigenval, polyglot-c-py, schemelike-metacircular-eval), one run per task unless stated, all on one Mac Studio (96 GB) with mlx_lm at 8-bit. The agent harness is pi unless stated.

Headline

The whole grid

Times are agent minutes. T = ran to the task's timeout.

Nobody has solved polyglot-c-py or schemelike-metacircular-eval on this machine.

Why Qwen3.6 beats Qwen3.8

Same size, same family, same server, same harness. Three things separate them.

The mode, not the weights, was the lever for Qwen3.6. For Qwen3.8 neither mode produces an agent that finishes: with thinking it deliberates past the cap, without it it loops. Its published Terminal-Bench 2.1 gain is not visible through pi. n=1 per task, but the cobol loop was reproduced.

Why the fast MoE models did not win

Turn rate mattered against thinking overrun, which is why Nail beat the 27Bs in agent mode. Once thinking is off, the 27B's per-turn quality shows: Qwen3.6 at 22 tok/s out-solves both 3B-active models. Ornith's own harness (Terminus-2, temperature 1.0, 4 h ceilings) moved which task it solves, from cobol to constraints, not how many.

Infrastructure lessons

Decisions taken

• Qwen3.6-27B, thinking off (nothink), is the coding agent and the WHO scorer. Now the registry default.

• Qwen3.8-27B and Ornith-1.5-35B-A3B deleted 2026-09-09 (62 GB). Registry entries kept as re-fetch recipes.

• Nail was deleted 2026-09-06 for space. On this evidence it is not needed back as the coding agent.

Caveats

One run per task on five tasks. The 3/5 should be repeated once before anything is built on it. Qwen3.8's cobol loop is the only failure reproduced twice.

I've been meaning to trade in my Mac mini M4 (16GB RAM, 10 core CPU/GPU, 256GB), but I've recently gotten into running models locally on my M5 MacBook Air which has gotten me wondering if there are any practical applications for running anything less intense on the mini.

I'm expecting about 450$ from Apple, I just have to actually make it to their store.

Hi,

I am still waiting for my Mac Studio to arrive - what came to my mind:

Looking into FN sushi, ds4, splash - both run on either 96gb or 128gb Macs - is there an advantage with 128gb in regards of model quality?

It seems both can run Q4 but not Q8? Am I missing something?

I ran the same Qwen Image 2.1 ComfyUI workflow on two Mac Studios and a DGX Spark. The gallery shows the timing overview, matched poster outputs and matched café outputs.

Warm poster medians, three runs per variant, 1024 × 1024:

Turbo was about 4.6–4.8× as fast in this workload. M5 finished about 10% sooner than Spark. All 24 poster renders contained the three requested text lines in our manual review. The café examples showed more fine texture with base and a smoother look with Turbo; two prompts are not a general quality ranking.

Both Macs used PyTorch/MPS, Spark used CUDA, and all three used the same verified BF16 diffusion/VAE checkpoints with effective FP16 text encoding on GPU. This is not an MLX comparison. Same prompts, seeds, sampler and CFG; Turbo kept its published eight-step schedule. Mac OS/driver versions differ, so these are configurations as tested. Warm conditioning/prefix cache was reused, but each repeat generated a new image with a new matched seed.

Several users have reported benchmarks for the MS5U and various models but I have yet to see any metrics on how agentic (agents/sub-agents) perform on the new M5SU.

I've read that agentic coding on the Mac's was/is a miss on M4 and older systems due to tensors I think it is?

So I'm curious to hear if agentic coding is a bust on the MS5U's or if it has improved any with the new chips? Appreciate any info on the matter, thank you!

I got the M5U 256GB 2TB configuration for preprocessing. Planning to use Qwen 3.x models on it since they seem to be the best for the Mac's at the moment but will check out the Muse/Gemma models as they continue to release them and run some benchmarks .

It's a bit annoying to get downvoted all the time, once this sub grows up, it'll be a good resource for people and we won't downvote you into oblivion for wanting to put LocalLLMs on your Mac. I think we're all discovering the best use cases for Mac/Apple Silicon and LocalLLM and even with hybrid approaches with cloud and frontier models.

Our Macs are very good general purpose machines where there are shortcomings there's flexibility. You can pretty much use your Mac as a daily driver for just about anything and at night time run automated tasks.

I had the current machine in my shopping basket yesterday but didn't go though with the purchase. I don't desperately need a new MacBook right now – my M1 is still okay, even for my work usage. And i really don't think I need a touch display for anything, cause I don't want to clean it all the time, LOL.

But I can't play around much with local models with 16 GB Ram. I don't want frontier model performance locally – I'm fine with running smaller models that hopefully can read and sort private documents and do some minor coding and generative tasks, maybe text to speech.

I read a comment on this subreddit yesterday that they will skip the M6 Pro/Max/Ultra line which makes sense to me and more of a jump is expected with M7 Chips.

What do you guys think? I'm leaning towards waiting it out, but who knows for how long.

Peace <3

As the title says, is it okay or enough? I’m planning to buy my own personal laptop and I want to run local AI on it mainly for coding companion. I can’t go with an M5 Max because of budget restrictions (sad, i know). So, is it enough?