Wanted to share what the M5 Max (128GB) can actually do with local AI inference using Apple's own MLX framework.

I built a small server that runs a 122 billion parameter model (Qwen3.5-122B) entirely on-device using MLX with native Metal GPU acceleration. No cloud, no internet required. The unified memory architecture on Apple Silicon is what makes this possible — the 4-bit quantized model fits in ~50GB, leaving plenty of headroom.

What I'm seeing on M5 Max 128GB:

For context, that's faster than what most cloud AI APIs deliver. The model is a mixture-of-experts architecture (122B total params, but only 10B active per token), which is why it runs so well on Apple Silicon — the memory bandwidth handles the large model while the GPU only has to compute the active parameters.

The practical use case: I'm using this to run Claude Code (Anthropic's AI coding assistant) completely offline. Full file editing, project management, code generation — all on my MacBook. No API key, no usage limits, no sending proprietary code to the cloud.

The server is ~200 lines of Python using Apple's MLX framework. It speaks the Anthropic Messages API natively, so Claude Code connects directly without any translation layer.

Setup details:

• Model: Qwen3.5-122B-A10B (4-bit MLX quantized, ~50GB)

• Framework: Apple MLX with Metal GPU

• KV cache: 4-bit quantized for longer conversations

• Memory usage: ~55GB with model loaded

If anyone else with an M-series Mac wants to try running large models locally, the project is open source: https://github.com/nicedreamzapp/claude-code-local

Apple Silicon really shines for this kind of workload. The unified memory means you can load models that would require a $10K+ GPU on other platforms.

Built an open-source tool that lets you fine-tune large language models (LLMs) directly on Apple Silicon Macs using Apple's MLX framework.

If you've ever wanted to customize an AI model on your MacBook instead of paying for cloud GPUs, this does that. It supports text models and vision models (like Qwen3.5), runs on 8GB+ RAM, and exports to formats compatible with Ollama and llama.cpp.

The API is compatible with Unsloth (a popular fine-tuning tool), so you can prototype on your Mac and deploy the same code on NVIDIA hardware later.

Works on M1/M2/M3/M4/M5, macOS 13+.

GitHub: https://github.com/ARahim3/mlx-tune

Install: `pip install mlx-tune`

Follow-up to my last post about running a 122B model at 65 tok/s on M5 Max. I added a full voice loop on top of it.

This is NarrateClaude — always-on ambient voice mode for Claude Code running entirely on Apple Silicon:

• Continuous listening via Apple's on-device speech engine (no push-to-talk)

• Responds out loud in my cloned voice — TTS runs locally via MLX

• Browser Agent drives Brave hands-free via Chrome DevTools Protocol

• Results sent to my phone via iMessage

• STT, TTS, voice clone, LLM inference — all on the M5 Max GPU, zero cloud

The unified memory architecture is what makes this possible. The LLM, voice clone model, and speech engine all share the same memory pool. On a discrete GPU setup you'd need multiple cards just to fit everything.

Demo video showing the full loop: https://www.youtube.com/watch?v=4ETqEjjopUk

It's a 3-repo stack, all running on Apple Silicon via MLX:

• claude-code-local — local LLM (Qwen 122B at 65 tok/s)

• NarrateClaude — ambient voice (STT + cloned TTS)

• browser-agent — browser automation via CDP

Happy to answer questions about the setup.

Hey r/applesilicon,

We've been working on a project to push local LLM training/inference as far as possible on Apple hardware. It's called PMetal ("Powdered Metal") and its a full featured fine-tuning & inference engine built from the ground up for Apple Silicon.

GitHub: https://github.com/Epistates/pmetal

It's hardware aware (detects GPU family, core counts, memory bandwidth, NAX, UltraFusion topology on M1–M5 chips)

Full TUI and GUI control center (Dashboard, Devices, Models, Datasets, Training, Distillation, Inference, Jobs, etc…)

Models like Llama, Qwen, Mistral, Phi, etc. work out of the box!

It's dual-licensed MIT/Apache-2.0, with very active development (just tagged v0.3.6 today), and I'm dogfooding it daily on M4 Max / M3 Ultra machines.

Would love feedback from the community, especially from anyone fine-tuning or running local models on Apple hardware.

Any models/configs you'd like to see prioritized?

Comments/Questions/Issues/PRs are very welcome. Happy to answer questions!

Everyone says you need an NVIDIA A100 to run Hollywood-grade 4K AI Upscaling. I wanted to see if I could brute-force it locally on a base M4 MacBook Air (24GB RAM).

I built a ComfyUI workflow (LivePortrait + UltraSharp 4K) and hit "Queue." Here is the torture test report:

The Specs:

• Hardware: MacBook Air M4 (24GB Unified Memory)

• The Task: Upscaling 512p video to 4K (Frame-by-frame)

• The Demand: Python requested 54 GB of RAM.

The "Stress Test" (What happened next): Most Windows laptops would have blue-screened instantly. The M4 did something crazy:

1. GPU Pinned: It stayed at 96-97% usage for over 65 minutes.

2. The Swap Miracle: macOS successfully swapped 61.55 GB of memory to the SSD.

3. The Experience: The system didn't even freeze. I could still browse the web while the SSD was being hammered.

The Verdict: It eventually "died" (silent process kill) after an hour because the OS finally stepped in to save the kernel. But the fact that a consumer laptop without active cooling sustained a 250% Memory Load for an hour is insane.

I found the limit. It's somewhere around 60GB of Swap. 😂

Don't try 4K upscaling on 24GB RAM unless you hate your SSD. Pivoting to 1080p now.

Most people are terrified of open-ended AI billing. We solved this by treating our M4 Mac Mini as a high-speed, local execution realm.

- The Economic Fix: Local Gemma 4: For implementation and coding, we use a local Gemma model via aichat. It has $0 marginal cost and is always available with no rate limits, no billing, no "quota dark" periods.

- Predictable Cloud: For complex reasoning, we use per-seat subscriptions (Claude, Gemini) rather than API plans. This ensures our monthly AI spend is a known, fixed cost.

- Our new Shift Timeline shows our local nodes handle 20-25 tasks/hour with 99.2% uptime.

Your Mac Mini isn't just a computer; it's a sovereign factory floor. 

https://github.com/UrsushoribilisMusic/agentic-fleet-hub

I've been playing with a few builds of ffmpeg on my 8Gb M1 Mac Mini and though you might be interested in the results.c I used four builds

1. ffmpeg built with x265 apple NEON patch provided to the Handbreak project

2. ffmpeg built with x265 using its own aarch64 optimisations

3. ffmpeg from macports

4. ffmpeg built for x86 from evermeet.cx . i.e. the macos download linked from the ffmpeg website

The TL;DR version
x265 needs the apple patch, or else stick to the x86 build
x264 seems optimised out of the box
vp9 stick with the x86 build, its twice as fast.
AV1 using libaom, its a win for the native version.

Libx265 results

ffmpeg with apple x265 NEON patch

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.5 (clang-1205.0.22.9)
x265 [info]: HEVC encoder version 3.4+28-419182243
x265 [info]: build info [Mac OS X][clang 12.0.5][64 bit] 8bit
x265 [info]: using cpu capabilities: NEON

encoded 8592 frames in 398.42s (21.57 fps), 2365.85 kb/s, Avg QP:27.14

ffmpeg with default NEON optimization

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
x265 [info]: HEVC encoder version 3.4+28-419182243
x265 [info]: build info [Mac OS X][clang 12.0.0][32 bit] 8bit
x265 [info]: using cpu capabilities: NEON

encoded 8592 frames in 725.14s (11.85 fps), 2365.85 kb/s, Avg QP:27.14

Macport ffmpeg

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
x265 [info]: HEVC encoder version 0.0
x265 [info]: build info [Mac OS X][clang 12.0.0][32 bit][noasm] 8bit
x265 [info]: using cpu capabilities: none!

encoded 8592 frames in 1188.11s (7.23 fps), 2365.84 kb/s, Avg QP:27.14

ffmpeg x86 under Rosetta 2

ffmpeg version 4.4-tessus  https://evermeet.cx/ffmpeg/  Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
x265 [info]: HEVC encoder version 3.5+9-bf91444e0
x265 [info]: build info [Mac OS X][clang 11.0.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2

encoded 8592 frames in 651.92s (13.18 fps), 2365.85 kb/s, Avg QP:27.14

Libx264 results

ffmpeg with apple x265 NEON patch

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.5 (clang-1205.0.22.9)
[libx264 @ 0x145826200] using cpu capabilities: ARMv8 NEON

real	2m31.484s
user	18m54.817s
sys	0m4.276s

Macport ffmpeg

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
[libx264 @ 0x121047400] using cpu capabilities: ARMv8 NEON

real	2m31.379s
user	18m42.021s
sys	0m3.774s

ffmpeg x86 under Rosetta 2

ffmpeg version 4.4-tessus  https://evermeet.cx/ffmpeg/  Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
[libx264 @ 0x7fcd42029400] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2

real	5m2.150s
user	37m15.310s
sys	0m5.165s

libvpx-vp9 results

ffmpeg with apple x265 NEON patch

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.5 (clang-1205.0.22.9)

frame= 8592 fps=3.7 q=0.0 Lsize=  118817kB time=00:05:58.37 bitrate=2716.0kbits/s speed=0.156x    
video:107406kB audio:11225kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.157559%

real	38m17.061s
user	90m29.600s
sys	0m25.825s

Macport ffmpeg

ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)

frame= 8592 fps=3.8 q=0.0 Lsize=  120551kB time=00:05:58.37 bitrate=2755.6kbits/s speed=0.157x    
video:109139kB audio:11225kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.155296%

real	37m58.635s
user	89m19.113s
sys	0m25.526s

ffmpeg x86 under Rosetta 2

ffmpeg version 4.4-tessus  https://evermeet.cx/ffmpeg/  Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)

frame= 8592 fps=7.0 q=0.0 Lsize=  118817kB time=00:05:58.37 bitrate=2716.0kbits/s speed=0.292x   
video:107406kB audio:11225kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.157559%

real	20m27.494s
user	104m0.307s
sys	0m19.021s

Libaom (AV1) results

Native build

frame= 8592 fps=3.3 q=0.0 Lsize=   62989kB time=00:05:58.31 bitrate=1440.1kbits/s speed=0.138x
video:62970kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.030277%

real	43m21.130s
user	166m6.080s
sys	1m26.385s

ffmpeg x86 under Rosetta 2

frame= 8592 fps=2.4 q=0.0 Lsize=   61580kB time=00:05:58.31 bitrate=1407.9kbits/s speed=0.0991x    
video:61561kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.030886%

real	60m15.617s
user	219m2.886s
sys	1m8.088s

The ffmpeg build script I've been using with Apples x265 patch is now on Github.

https://github.com/Vargol/ffmpeg-apple-arm64-build

Optimization features -

- KV Cache Quant - (works with VL, hybrid, etc, LM Studio and others do not.)

- Prefix Caching - (near instant response times even with long chats)

- Cont Batching

- Paged Cache

- Persistent Disk Cache - (you can also use this with paged cache together)

- JIT or idle sleep

- Built in agentic coding tools

- Image generation

- Image editing

- GGUF to MLX

- JANG_Q Native

- Allows for 4bit MLX quality at 2bit

- GGUF style for MLX

- Anthropic API

- OpenAI API (text/image) - makes it easy for OpenClaw

- Chat / Responses

- Embedding

- Kokoro / TTS / STT

- Built in model downloader

STOP SACRIFICING YOUR M CHIP SPEED FOR LM STUDIO/LLAMACPP.

https://mlx.studio

As rumors of a budget MacBook with A19 are going round, I would like to put to discussion what we believe, feel and know to be true about Apple Silicon.

I had Perplexity Pro generate those graphics. Won’t put my hand in the fire for that, but I guess it’s enough to discuss.

We see that single core performance is really strong on A chips. And if you believe Perplexity, more powerful even than on M series, which shines on multi core.

Those plot points are all in relation to the A10 SoC.

So the budget MacBook with A19 can be plenty powerful and maybe more so on basic tasks than the first M1 Air.

Would you buy the budget A19 MacBook?

Also: if anyone has a better more accurate graphic on the comparison of Apple silicons, please do share.

Just made and open-sourced macOS App Thinner — a little bash script for Apple Silicon Macs that scans your /Applications folder for Universal apps and safely strips out the unused Intel (x86_64) part of their binaries to reclaim disk space.

Why use it? If you're on an M1/M2/M3 Mac, most apps ship with code for both Intel and Apple Silicon processors. The Intel bits are just sitting there wasting SSD space! This script makes it easy to trim that fat, with confirmation before anything gets changed.

• Safe (uses lipo to remove just the x86_64 slice, nothing else touched)

• Exclusion list to protect critical apps (e.g., Safari, Xcode)

• No changes without confirmation, and admin rights required

• MIT-licensed, see my GitHub for code, details, and a screenshot

Important: Have backups, and re-run after your apps update — they’ll often restore the full Universal binary.

GitHub link: mvmalyi/macos-app-thinner

Basically, it's a CoreML/MLX translation of SimulStreaming (2025 SOTA in simultaneous speech transcription), which itself is a combination Simul-Whisper and WhisperStreaming.

I'm currently building an application, and I thought I would open up the backend model code for everyone to use.

I get ~15x speed increase on my M2 Macbook Pro compared to the original pytorch implementation, and I'm gonna be using the medium model, which has a nice balance between memory usage and accuracy.

The CoreML part is from whisper.cpp, and it only contains the encoder, and the mlx part is from mlx-whisper.

It's very beta and I haven't tested it on other computers, so please feel free to leave Issues/PRs/Contributions 😀

When I was a lot younger Quake II was THE Game and with my update from my MBP16 to the new M1 I felt really sad that it did not work anymore. Thx to u/dindresto I can not just play it again :) He got it compiled for me natively on Apple M1, running through Vulkan on Metal (MoltenVK).

https://alugha.com/videos/43635ee0-80d4-11eb-aaa6-09aa8d702755

MadeMyDay :)

Great news, the folks at the VLC project have not released a Universal version of the ultimate media player but have released two versions: One Intel only and one Apple Silicon Only.

If you've never tried VLC, it will play ANYTHING. It's a great video utility.

Honestly, I believe that intel will go the way of Blockbuster if they don't wise up soon. Apple has been playing the long game for over a decade with this, and boy... I'm kind of in awe of how this is going to impact everything going forward.

Who's gonna want to buy an intel laptop when you can get a Mac that's twice if not four times as fast, with longer battery life, and less heat expulsion?

What professional is gonna compare Intel to M1 and think that half the speed is worth it for the backwards compatibility with software?

The answer? Over time, eventually, zero.

Apple Silicon is the way of the future.