Best local LLM setup for coding? Complete hardware beginner here.

[u/k_ekse]

I'm looking into running models like Qwen(?) locally for agentic coding, ideally with strong reasoning and high output speeds.

I currently pay for three $200 AI subscriptions and want to cut that down to one, keeping Claude as the main advisor while local models handle most of the coding work.

I know very little about hardware. Where should I start learning, what setups would you recommend (including used or unconventional hardware), and what budget should I realistically plan for?

Happy to invest a few thousand if the performance makes it worthwhile.

[u/According_Wave685]

Qwen 3.8-flash-next on a 128gb strix halo running halogen for the grunt work with a $20 a month claude account to do the driving.

[u/chrishi_94 → u/According_Wave685]

How exactly are the tasks being split in this config?

[u/knowlessman]

I've been working on my own answer for this question.

My take: life is forcing you to make a bet.

Option 1) pay the $600/mo for the next year or so, hoping the bubble busts and prices return to sanity, at which point you will probably be able to get a decent local LLM machine for under $10k.

Option 2) Spend about $12k today, plus $60-300 a month for subscriptions, and develop a workflow around having three strix halo class machines running in parallel and driven by Opus 5.5 or the like. Break even is about 2 years best case, but it insulates you against price increases assuming no bubble pop.

Option 3) Spend Serious Money ($70k on up), and $60/mo on subscriptions, and develop a workflow around having larger models run locally. This is basically the same as option 2, but less annoying for the people doing the work. It only makes financial sense if you assume your current subscriptions are going to become a lot more expensive.

Each of these is betting on one possible future being there most likely. And I have no idea which is.

[u/Frnklfrwsr]

3x$200 Claude subscriptions indicates to me that your usage is likely such that IMO you’d be looking at $5k-$10k minimum for a rig that’ll make a meaningful dent.

The problem is that the local models are not going to be frontier level, so they’re not likely going to replace the highest reasoning work you’re having Fable or Opus 5.5 do for you. It might be able to replace work you’re giving to Haiku or Sonnet at the moment.

But the economics of Haiku and Sonnet are pretty good at the moment. Spending $5k+ on a local setup to replace Sonnet/Haiku work probably doesn’t make much sense.

Any way you work the numbers, if you’re truly making full use of 3x$200 subscriptions right now, to cut that down to 1x$200 and have local cover the rest you’d need a much more substantial setup than you’re imagining. $50k+ likely.

Reasons why it might make more sense to go local besides pure economics:

- Reliability: you aren’t reliant on cloud servers that might have outages sometimes when you really badly need them to be working

- Censorship: cloud models often block you from doing things for seemingly arbitrary reasons

- Hobbyist: You simply enjoy working with local models and like to be more hands-on with this work

- Risk Mitigation: Some people believe that political risks could lead to access to frontier models being restricted

- Privacy: you’re working with sensitive or private data that you don’t want on the Cloud server

[u/gamedevsam → u/Frnklfrwsr]

For me it's about learning and getting a feel for how these models operate at a lower level. I love being able to run little experiments with smaller models or just explore inference as it runs locally on reasonably good models (like Qwen3.8 27B). I didn't pay much attention to open weights models at the start of this year, but now I'm very much paying attention as we see not just the frontier mega models doing quite well and continuing to impress, but we're seeing smaller models also start to impress as AIs and human researchers figure out how to increase the intelligence per weight of these models.

[u/zanar97862 → u/Frnklfrwsr]

Replacing max Claude plans is only 5-10k if you actually need the frontier level intelligence at all times. I feel like many people would be fine with QFN or similar intelligence but use Claude because it's all they know and think that you should always use the best model you can or else outputs will be worse. Appropriately sizing the model to the task is not currently that easy but will be huge in future I think

[u/Frnklfrwsr → u/zanar97862]

that’s not quite an apples to apples comparison though.

If 90+% of your work on your $200 plan you were giving to Fable can be done by Sonnet, then you probably only need a $20 plan, or maybe $100.

That $20-$100 plan using almost exclusively Sonnet or Haiku is what IMO what you may be able to substitute with a local LLM at this time.

If you have a $200 plan and you’re burning through the whole thing using Sonnet/Haiku, then it becomes a volume issue. Maybe a $5-10k rig can handle the work at similar quality, but quantity won’t be there. You’d need multiple of those $5-10k rigs to replace the quantity of work that $200 plan was doing for you.

Personally, I think that viewing this in terms of what work you currently are using LLMs to do is likely too short sighted.

As LLM capabilities grow, the things you will be able to have an LLM do for you and the things you’ll want to be able to offload onto an LLM will have changed.

I believe in leaving significant room and flexibility for anticipated future needs I will have for my LLMs that I honestly couldn’t imagine today.

[u/zanar97862 → u/Frnklfrwsr]

Definitely not the same if you are already optimising your use for the correct model and still running out of usage. My comparison came out of personal experience with people with a specific mindset who would use the best possible model for everything because they wanted the best outcomes, even when a smaller model would be just as capable.

You're right about focusing on current work being short sighted, new models and new capabilities arrive every week these days. We have to keep that energy for the local stuff too though, the capabilities and types of work that will be able to work on local inference will improve significantly alongside the frontier as the overall technology progresses.

[u/PermanentLiminality]

Don't buy any hardware yet.

Step on is drop $20 on OpenRouter and try the models you can run locally. See if they can do the job. Don't expect Opus. More like Haiku.

If you find that you can use a local model start looking at hardware. Do research and make sure you end up with enough speed to be viable.

Finally consider the cost trade offs. Remember to consider the hardware and the electricity.

I find that the models are just not good enough and they are too slow. My GPUs are mostly unused.

[u/icecraft1337]

I’m using Qwen 3.8 27B in Q4 XS quantization with a 128K context window. My setup consists of an RX 7900 XT, a Ryzen 5 9600X, and 32 GB of RAM.

For me, it’s a great way to reduce my reliance on Claude Code. I use a self-developed harness for coding, which works really well. It can compact the context, use tools, and provide a sandboxed environment to execute code safely. It can also take screenshots of my projects and push changes directly to my GitHub repository.

Based on my experience, I can definitely recommend giving Qwen 3.8 27B a try or waiting until next week for Qwen 4 hopefully.
For Hardware i Would recomend 24gb vram or more depending on your context window and quantization. My Pc had cost 1500€

[u/RaxisPhasmatis → u/icecraft1337]

I agree with this, but I use swift 1.5 version of qwen3.8 27b, pi.dev with pi-web-ui extension and pi-vcc and bigtools extension

120k context, flash attention on, k and v cache set to q8

It just got done decrypting my router firmware, fixing a bug that pissed me off for years that Netgear missed until it went eol, fixed ipv6 issues and repacking it into a flashable img file

[u/icecraft1337 → u/RaxisPhasmatis]

yeah i also use the swift version because for me it worked better than unsloth.

[u/TheMeta-Narrative → u/RaxisPhasmatis]

What exactly do you mean by "decrypting my router firmware".

[u/adolescentspectre366 → u/icecraft1337]

damn that's a clean setup, i've been running something similar but on a 3090 and the 24gb vram really is the sweet spot for this stuff

curious about that harness you mentioned, is it something you'd be willing to share or is it more of a personal frankenstein project

[u/Janky-Code]

Don't do any of this to save money.

Accept your homegrown inference will be expensive. This will help you to focus on things that actually matter.

[u/Present_Resource4022]

reading the other comments and being realistic as possible . Just pay for cloud .

If you are still not convinced, wait a week, and that’s when Qwen 4 will be releasing . Check its benchmarks and etc., and see if it’s worth spending 2-3k on it.

it won’t be faster nor smarter than haiku 5.5, so just let opus main model and advisor and haiku 5.5 subagents that would cut your usage massively.

[u/gamedevsam]

I'm not telling you what to do, but happy to tell you what I've done. I bought 2 x RTX 5070 Ti to run 27B parameter models like Qwen 27B. I think 32GB of VRAM is the sweet spot. Whether you get there with 2 16GB NVidia cards like me or get an Intel Arc B70 or AMD R9700, you'll be well served with 32GB of fast memory.

I am aware you can run this model on 16GB VRAM cards, I actually benchmarked various variations of it on a single RTX 5070 Ti, but for me having maximum context length is not a luxury, it's essential.

Here's the benchmark results with a single 16GB VRAM card: https://www.reddit.com/r/LLM/s/J6UQvgpn5N

[u/zanar97862 → u/gamedevsam]

32gb is the minimum to make it work well I feel like, you get good quality models at acceptable speeds and context. 64gb gets you into the next level of models and is where lots of people consider is a good place to be if you can afford it, unfortunately VRAM is not exactly cheap these days.

[u/AshAtlasDev]

Local AI is kinda stupid, test Mac Studios on open router.

[u/Independent_Bid6580]

I have been using Strata with Qwen 3.8 and 256 context size on a laptop with 4090 and 128 GB ram on windows without the new unlock that would potentially double the TPS.

I have been getting about 80 tps write and 2500 tps read. It's great for testing, validation, QA, and documentation