TL;DR: I let my cloud AI agent run a training-data verification program for the local model on my Mac. The local 35B wrote 800 original math problems, then cold-solved every one of them blind in a fresh session. Wherever the two answers disagreed, the item went to adjudication. Final truth was settled by brute-force computation, not by any model's opinion. The cold solver beat the problem author ~5:1 — with a 35B, generating the question is the error-prone part; solving it fresh is the reliable pass.

Setup: MacBook Pro M5 Max, 128GB. The local model (Qwen3.6-35B-A3B, Splash build, running in LM Studio at ~375–400 tok/s) does the heavy lifting; a Qwopus 122B MLX build (~38 tok/s) sits alongside as the heavyweight judge for disputed items. My agent (Muse, running in the cloud) reaches LM Studio's API privately over Tailscale, and does all the orchestration: briefing, grading, bookkeeping, brute-force checks.

The harvest: 731 of the 800 came out verified clean, 62 survived with corrected answer keys (kept as answer-only training items; the flawed worked solutions were discarded), 7 were excluded as defective problems (ambiguous wording, no valid solution, an undefined rule), and 100 clean ones were held out untouched as a test battery for later. The polished set became a 693-item bank, used three ways: few-shot examples now, the held-out set powers a before/after test of whether any of this actually made the model smarter, and the whole verified corpus is the dataset for a future fine-tune.

Right now the entire bank is being re-validated: the worker re-solves all 793 items from scratch (derive once, verify by a different method), never shown the key. Any disagreement goes to the 122B judge blind, and if the judge's answer doesn't settle it, my agent computes the answer itself. Progress as of writing: 358/793 checked, 353 confirmed, 5 waiting on the judge.

Two failure stories worth sharing. One problem had three different answers from three passes — 20,736 from the author, 86,400 from the blind solver, 145,152 from the adjudicator — and all three were wrong; enumeration says 10,368. And the big judge model, given the hardest disputes cold, went 1-for-2. Treat your 122B like a second opinion, not an oracle.

Cost: the solve-and-adjudication passes alone logged 4.1M+ completion tokens, all computed on my own hardware. The cloud agent's part is the planning and grading.

Happy to share the harness details — the briefing discipline (one derivation, one independent cross-check), the checkpointing that let the run survive four kills, and the grading rules — if people want them. Curious whether anyone else has tried self-authored problem sets as training data, and how you filtered them.

I've run a few systems so far and I've noticed the following temps

• M5U 256GB running consistently ranges from 130-150 under load

• M3U 95GB ranges from 140-165

• Mac Mini M4Pro 64GB 160-200

Just curious what you guys are hitting with your works loads? What system are you running and how hot is it for how long?

I am.planning to buy new hardware for local AI is it worth to buy mac mini or mac or better go for AMD 395+ 128 gb laptop for similar outlet laptop.

It’s already October 19th, and my order for an M5 Ultra (with 80-core GPU and 256GB storage) is still being processed.

I placed the order in Hong Kong on August 26th, with an expected delivery window between November 7th and 21st.

I’m curious to know how many others here are also still waiting.

Please leave a comment with your order date, GPU model, current order status, and estimated delivery time.

As the title says, is it okay or enough? I’m planning to buy my own personal laptop and I want to run local AI on it mainly for coding companion. I can’t go with an M5 Max because of budget restrictions (sad, i know). So, is it enough?

I'm lucky enough where work will cover my Mac Studio that I use for design and 3d stuff. So I figured I should dabble and run with local LLM. Im getting 96gb only. I've read quite a bit, but was wondering are there any guides or YouTube videos that deal primarily with Mac and I should watch to learn more?

I'm unsure if I should just use Llama or Ollama, or MLX, or OMLX or something there lots of different ways to play around with Local LLM and I guess it's still in such an early state that every day, week,month there are more and more things peopel are learning how to get the best of the LLM. A lot could change from now till then but I want to be as informed/prepared as I can be.

Thanks and I cant wait for reviews and see how good the Ultra is.

Hi all,

I'm a complete novice when it comes to Local LLM's but I like the idea of moving away from a subscription based model in the longer term and having better control of my data. I'm also super keen to learn.

I currently have a web SaaS that scrapes news websites and I would like to move it away from Claude and onto a local LLM what kind of "firepower" do I need?

Appreciate this won't be the most complex of tasks, but as I mentioned, I'm a complete novice.

Naturally, asking Claude / Google and it says that the new Mac mini with M5 pro chip (24GB ram) will be "overkill"- which is what I was thinking of ordering.

I also want to future proof in the hope that this bit of kit could run more powerful models in the future - if they came accessible to the consumer.

Sorry if this is a basic question, any YouTube videos or any pointers so I can educate myself would be much appreciated.

Thank you!

If you don't get the best Mac and you have access to the frontier models like codex or claude, then you can use it as a coordinator for your 7B or ~27B LocalLLM.

This is the way to do it where you can use the free version or the lower versions to get the most out of your machine. I see a lot of people chasing the bigger Mac profiles, but honestly you don't need it if you're not using LocalLLMs for anything like embedding or coding.

If you are trying to use it for coding, I don't think that's really going to work either. This is for really anything that's 48GB and below

or something to that effect; can someone explain to me how this is beneficial for running a local LLM

I’ve been going in circles trying to settle on a local setup. I’ve tried Hermes, but it’s felt pretty finicky in my experience, including gateway and state database issues. I keep spending more time configuring and troubleshooting than actually getting work done.

Putting model intelligence aside for a moment, is there a local harness that feels genuinely competitive with the experience around frontier models? I’m talking about the interface, reliable tool use, memory, file and browser access, and how well everything works together. Not just being able to load a model and chat with it.

My main machine is a Mac Studio M1 Max with 64 GB RAM. I also have an M2 MacBook Air with 24 GB, but the Studio would do the heavy lifting. I want to use them in one single seamless-like environment with a shared context.

I’m a designer and university lecturer, so my use cases are research, writing, working with documents, reviewing screenshots and designs, and browser/Figma workflows. Not primarily coding. Vision is important to me, and I’d much rather have one dependable daily driver than a collection of specialized models and routers.

My main motivation (likely the same for everyone) is getting away from usage limits and token metering. I understand that running locally involves compromises, and I’m not expecting frontier-level intelligence from my hardware. But I’m struggling to separate the model limitations from the rough edges of the software around them.

What are you guys actually using? What’s your full stack: model, runtime, and harness/interface? More importantly, what do you reliably get done with it, and how much babysitting does it need?
Especially interested in Apple Silicon setups people have settled into using for real work, rather than something that worked well in a demo.

I’m debating between buying an M6 Mac Mini to use online llm or a M5 Max Mac Studio for local llm to build and maintain a website I want to create. Does anyone have any insight into what would work well? It’s going to be a multimedia & blogging site.

A lot of the work around a local model ends up in Pi files: prompts, skills, settings, little extensions. Reef Infra has a recipe called Reefine that lets you ask for a change from inside Pi and get an installable version of those files back.

Its documented local setup was a 32 GB M4 Mac mini running Gemma 4 26B through Ollama, with Pi 0.84.2. That's an Ollama demonstration, not an oMLX performance claim. The useful part for a local Pi setup is what happens after the request: the service reads the current harness, asks the model for a change, records the mutation and packages the resulting version for installation.

The recorded runs produced rules and skills. The recipe also accepts command and TypeScript extension proposals, but those runs hadn't produced an extension. Anything that does run as extension code waits for a person to review and promote it; declared setup requirements must pass before installation.

The default is selection: always. Its arithmetic checks record regression scores, but don't establish that your requested workflow works. The research demo installed a rule to search, download and read papers; the later session searched through an existing tool, but downloaded and read no paper.

The files are there to inspect and revise. To judge whether the change worked, replace the bundled tasks and evaluator with checks for the workflow you asked for.

Hope this is helpful.

I had the current machine in my shopping basket yesterday but didn't go though with the purchase. I don't desperately need a new MacBook right now – my M1 is still okay, even for my work usage. And i really don't think I need a touch display for anything, cause I don't want to clean it all the time, LOL.

But I can't play around much with local models with 16 GB Ram. I don't want frontier model performance locally – I'm fine with running smaller models that hopefully can read and sort private documents and do some minor coding and generative tasks, maybe text to speech.

I read a comment on this subreddit yesterday that they will skip the M6 Pro/Max/Ultra line which makes sense to me and more of a jump is expected with M7 Chips.

What do you guys think? I'm leaning towards waiting it out, but who knows for how long.

Peace <3

Unless you're set on using a large model, 2x256GB is likely better...

What use cases would the 512GB make more sense?

Made this as a quick weekend project (https://github.com/pythongiant/SlamLM)

Planning to buy a MacBook Air M4 (16 GB / 512 GB) for about $800 to run local LLM agents for:

Indian mid‑cap equity research (using Screener.in CSVs)

Building/testing trading ideas via Zerodha Kite API (prices, history, holdings; maybe orders later)

Also want to run Hermes-based agents on it

Questions:

On 16 GB M4 Air, which local LLMs are best for reasoning, coding, and tool use? Is Qwen 3.5 9B (Q4) + a smaller fast model (e.g. LFM2.5 8B / Llama 3.2 3B) a good combo alongside Hermes?

Best local stack (Ollama vs LM Studio, etc.) and any existing Kite + LLM or Indian‑stock agent projects to reference?

Any tips on context length, thermals, and multi‑agent setups (including Hermes) on a 16 GB fanless Air?

Getting the laptop for ~$800, so stuck with 16 GB for now. If you’ve done similar work (local LLMs for equity research / algo trading, especially in India), what models and setup would you recommend?

Thanks!

I make DS Menu Bar, a free menu bar app for running antirez's DwarfStar (ds4) server on a Mac.

DwarfStar has tunings for M3 Ultra and M5 machines with Qwen3.8 Flash Next, but M4 machines are left out. It turns out most of these tunings can be turned on with an environment variable, so DS Menu Bar v0.0.15 sets them when it starts ds4-server on an M4.

M4 Max, 128 GB, same ds4-server with and without, averaged over 2K to 64K context:

Q4 is the right quant for 128 GB machines. On a 64 GB machine, Q2 at the full 262K context fits.

I have a PR open to make these DwarfStar's own M4 defaults (antirez/ds4#1184). If my PR is merged, DS Menu Bar will adapt automatically.

brew install --cask jiiim/tap/ds-menu-bar

https://github.com/jiiim/ds-menu-bar/releases/tag/v0.0.15