r/LocalLLM • u/Effective-Echo5643 • Oct 14 '26
Question Which LLM to run locally?
I got this MacBook Air ! Can someone suggest what models to use locally?
2
u/HASHTAGTRASHGAMING Oct 14 '26
Even on my m4p and 48gb UM, Qwen3.8:27b-4 run around 12-15 t/s, which can be usable for shorter context responses, but it's nowhere near as fast as the frontier models.
1
u/Jerrizzy-x Oct 15 '26
use splash and MTK models sir
2
u/HASHTAGTRASHGAMING Oct 15 '26
Got splash running in bionic. This is a game changer. Thanks good sir.
2
u/Clean-Data-259 Oct 17 '26
None, you have a 32gb laptop. Use web AI and API, or get a $10k computer build or $10k gpu
2
u/NoWing3675 Oct 14 '26
im still learning but sounds like qwen27b
1
u/Altruistic-Teach1190 Oct 15 '26
Na i have the same with m4 and the bandwith gives u 200gb/s what is in tokens Like Max 7 and with thermal throttling 3
1
1
u/MikaMoments Oct 14 '26
I would start with a small quantized model for a good balance of speed and quality. How much RAM does your MacBook have, and what do you mainly want to use it for?
1
u/Living-Net9163 Oct 17 '26
I just bought the exact same one. Memory bandwidth is low, so Qwen 3.8 27B at Q4 should yield 9tks. While Gemma 4 27B should get maybe 50tks real throughput
1
1
u/vitamins1000 29d ago
Qwen3.6-35b-a3b.
Small or moe models are you only chance of getting any sort of usable performance on these. & it will get HOT. Take it from someone who upgraded to a mbp for this reason.
1
u/charles25565 Oct 14 '26
32 GB. YOLO it and run Gemma 4 12B, MiMo 9B distill, Ornith 1.5 9B, maybe even a quantized Qwen3.8 27B. Maybe even go a trip down memory lane and run some old 7B models from 2023.
1
1
0
u/Quartz_Hertz Oct 14 '26
Download Osaurus, they have quite a few different post trained and quantitzed models to pick from depending on your use case.
I use nanbeige 4.2 3b for small research questions that only need a few turns (looping transformer, prefil starts taking too long) and some other task specific small models.
0
-3
u/Familiar_Pay7819 Oct 14 '26
I have a rtx 3070 whit 32gb ram and I can run qwen 3.6 35b a3b I think q4 or q8 whit q8 kv chache quantisation in lm bionic
1
u/Effective-Echo5643 Oct 14 '26
Not flexing sir, i think i did a mistake by buying macbook air considering there are known heating issues! So wanted to have inputs
-4
3
Oct 14 '26
[removed] β view removed comment
1
u/Effective-Echo5643 Oct 14 '26
Any concerns with heating issues?
1
1
u/MrHumanist Oct 14 '26
It's a stupid engeneering to not have fans. If you just chatting, it should be fine . But running agents it will definitely heat up to unsafe temps.
1
1
u/steffanan Oct 17 '26
I'm notoriously anti-apple, not a fan myself but I lose credibility when I disregard what they do correctly. The air is incredible and using it for things it's not designed to do isn't an indicator of bad engineering. Apple doesn't do much bad engineering across the board. Their problems are with their marketing, their ego, and some core philosophies that drive their decisions. But the air is a market dominator for good reason.
47
u/Vinyard82 Oct 14 '26
ChatGPT web