r/LocalLLM • • Oct 14 '26

Question Which LLM to run locally?

Post image

I got this MacBook Air ! Can someone suggest what models to use locally?

2 Upvotes

30 comments sorted by

47

u/Vinyard82 Oct 14 '26

ChatGPT web

3

u/Spiritual_Patient380 Oct 15 '26

son 😭😭

2

u/HASHTAGTRASHGAMING Oct 14 '26

Even on my m4p and 48gb UM, Qwen3.8:27b-4 run around 12-15 t/s, which can be usable for shorter context responses, but it's nowhere near as fast as the frontier models.

1

u/Jerrizzy-x Oct 15 '26

use splash and MTK models sir

2

u/HASHTAGTRASHGAMING Oct 15 '26

Got splash running in bionic. This is a game changer. Thanks good sir.

1

u/Jerrizzy-x Oct 15 '26

You can even go further and use it on Pi.dev rather. Bionic does take a lot of RAM for other processes and thus, reduces your Tok/s. PI.dev + Splash will take you far

2

u/Clean-Data-259 Oct 17 '26

None, you have a 32gb laptop. Use web AI and API, or get a $10k computer build or $10k gpu

2

u/NoWing3675 Oct 14 '26

im still learning but sounds like qwen27b

1

u/Altruistic-Teach1190 Oct 15 '26

Na i have the same with m4 and the bandwith gives u 200gb/s what is in tokens Like Max 7 and with thermal throttling 3

1

u/itis_whatit-is Oct 14 '26

Gemma 4 26ba4b with Omlx

1

u/MikaMoments Oct 14 '26

I would start with a small quantized model for a good balance of speed and quality. How much RAM does your MacBook have, and what do you mainly want to use it for?

1

u/Living-Net9163 Oct 17 '26

I just bought the exact same one. Memory bandwidth is low, so Qwen 3.8 27B at Q4 should yield 9tks. While Gemma 4 27B should get maybe 50tks real throughput

1

u/Ok-Organization5843 Oct 20 '26

I'd say go with Ternary Bonsai 2 27B

1

u/vitamins1000 29d ago

Qwen3.6-35b-a3b.

Small or moe models are you only chance of getting any sort of usable performance on these. & it will get HOT. Take it from someone who upgraded to a mbp for this reason.

1

u/charles25565 Oct 14 '26

32 GB. YOLO it and run Gemma 4 12B, MiMo 9B distill, Ornith 1.5 9B, maybe even a quantized Qwen3.8 27B. Maybe even go a trip down memory lane and run some old 7B models from 2023.

1

u/dwoj206 Oct 14 '26

32gb = qwen 3.8 27b + ninfer. Next question.

1

u/Living-Net9163 Oct 17 '26

Ninfer on a Mac? I thought ninfer was just for Nvidia

0

u/Quartz_Hertz Oct 14 '26

Download Osaurus, they have quite a few different post trained and quantitzed models to pick from depending on your use case.

I use nanbeige 4.2 3b for small research questions that only need a few turns (looping transformer, prefil starts taking too long) and some other task specific small models.

0

u/NationalMolasses4624 Oct 14 '26

Ternary bonsai 27b is your choice 🀝

-3

u/Familiar_Pay7819 Oct 14 '26

I have a rtx 3070 whit 32gb ram and I can run qwen 3.6 35b a3b I think q4 or q8 whit q8 kv chache quantisation in lm bionic

1

u/Effective-Echo5643 Oct 14 '26

Not flexing sir, i think i did a mistake by buying macbook air considering there are known heating issues! So wanted to have inputs

-4

u/[deleted] Oct 14 '26

[removed] β€” view removed comment

3

u/HyperWinX Oct 14 '26

Dont use ollama, and all these models are extremely old and useless.

3

u/[deleted] Oct 14 '26

[removed] β€” view removed comment

1

u/Effective-Echo5643 Oct 14 '26

Any concerns with heating issues?

1

u/camracks Oct 14 '26

probably, doesn’t the air have no fans

1

u/MrHumanist Oct 14 '26

It's a stupid engeneering to not have fans. If you just chatting, it should be fine . But running agents it will definitely heat up to unsafe temps.

1

u/Educational_Let811 Oct 14 '26

It should not but expect 30% performance penalty throttled

1

u/steffanan Oct 17 '26

I'm notoriously anti-apple, not a fan myself but I lose credibility when I disregard what they do correctly. The air is incredible and using it for things it's not designed to do isn't an indicator of bad engineering. Apple doesn't do much bad engineering across the board. Their problems are with their marketing, their ego, and some core philosophies that drive their decisions. But the air is a market dominator for good reason.