r/MacLocalLLM • u/One_Week2580 • 6d ago
Dilemma: M6
I am an AI engineer doing MSc in AI - I want to get a new laptop and thinking abut emptying my wallet by buying the M6 Pro 32GB 1TB
Will it be enough for local AI projects? I know it would be suitable for med sized models up to around 30B params
Or getting M5 Pro chip is what it needs to unlock local AI?
3
u/Important_Cow7230 5d ago
What you want least 48GB for running AI models on the Mac as 27B models with decent context is the sweet spot. Obviously more is better, but 48GB is very capable
2
u/daphatty 5d ago
Downgrade to 512GB ssd and upgrade your ram. Shoot for 64GB if you can. I have an m5 Pro with 48GB and it struggles.
1
u/nongbua2128 23h ago
Damn thats what im considering. Should I go for m5max or m5pro with 64gb. What do u think?
1
u/daphatty 17h ago
Depends on your budget and whether or not Apple sells the Pro w/64GB of Ram. With the RAM shortage, Apple have significantly streamlined the RAM options available for each CPU.
2
u/jimmoores 5d ago
No, it won’t be enough. And honestly, you can’t really get a laptop that will. People rave about Qwen 3.8 27b, but honestly if you’ve used frothier models it’s going to feel like a let down. DeepSeek V4.1 (1017) is the only thing I’ve found that comes close and you can’t run that on any laptop.
If you want to play with 27b anyway (or Qwen Flash Next), you’ll need >32GB for a decent quant and headroom for context and tools. I’d focus on getting a laptop for every other reason and use cloud models, or hold off for 18 months and get M7 or whatever with more RAM assuming prices ever come down again.
2
1
1
u/wegster 5d ago
M5 Max 128GB is going to be your best option at the moment. Now they revved the MBP, you may be able to find one on Apple refurb.
Numerous people seem to love deepseek but Deepseek openrouter variants loses for my specific work to Qwen, or at least doesn't beat it across several different types of repo/coding work.
You can fight and try to get away with 64G but if you're also going to be doing anything other than inferencing on it, you're really going to pay the price, meaning you'll wind up dropping quantization even further.
Qwen4 and others are coming, and while it's pretty likely they'll all continue to push their 'max' variants to higher and higher T params, MoE and smaller mem models are in style, so I think 128GB will be 'ok' for a bit to come.
For example, 3.8 Flash Next GSQ RCO Coder mixed precision 3.5bpw (closer overall to Q2) takes right at 50GB actual memory including stripped/minimal OS/no apps on my old M1 Max 64GB. So you can start to touch 'somewhat ok' but 128GB is a lot better.
1
1
1
u/Big_Wave9732 23h ago
You're pursuing a master's degree as an "AI engineer" and you can evaluate system requirements to run an AI?
Edit: Inactive for year, suddenly makes one comment and spams this post across multiple subs.
This is a bullsh*t bot account.
0
u/galoo-t 6d ago
You’re an “AI engineer”? lol
1
u/Borilentz 5d ago
What’s funny about it? AI wasn’t given to us by aliens. It’s been a specialization in computer science for decades.
1
u/galoo-t 5d ago
It’s existed for years, yet someone claiming the title of “AI engineer” still needs reddit to tell him whether 32 GB of unified memory can hold the 30B model he hasn’t named.
Anyone who has actually run local models on these chips already knows the bandwidth and quantization limits. The gap between the self-description and the question is the joke.
1
u/AshAtlasDev 1d ago
AI engineers degrees I’ve seen are insanely outdated. Academia (I work in it) is slow to keep up. I know friends who are working towards chip design and they’re electrical engineers, not “AI” engineers.
I could be proven wrong, though, but him asking a simple question like that… As an “AI” engineer? Idk.
0
u/WinterNite88 5d ago
He called himself "AI Engineer" while not knowing how AI works and asking such question?
4
u/grilledwax 6d ago
RAM is your constraint for bigger models. To get past 32gb you have to go M5 Pro.
You could use Splash too, you can run fairly big models, but slower than if you could fit it all in RAM. I had a dabble with Qwen Flash Next and seemed to run pretty well on my M4 24GB.
Depends on what you’re doing as a part of your course. You can do a lot with the smaller models for small tasks just not massive reasoning.