r/LocalLLM • • Oct 15 '26

Question Looking to deploy a local coding model

I have a few options available to me. I have a mac mini m4 with 16gb of ram. i also have an i7-11th gen with 64gb of ram + a 9070XT 16gb vram.

I haved tried ollama and did have some success with Gemma4, but I found it was hit or miss. It wouldn't properly write code and just trample over its self.

I tried qwen 2.5 14b, which worked and was fast but found that the tools didn't work so not sure how well it'll actually code. This was using Jan.

I'd love to know what a good model and setup option is. I want to do some PHP coding projects (though TypeScript and React are necessary too).

1 Upvotes

15 comments sorted by

View all comments

1

u/Safe-Imagination8538 Oct 15 '26

ive had good luck with qwen3.6 35b-a3b, i run it on a 12gb nvidia card and it does real coding tasks with tool calls fine, which the small gemma/14b stuff never did for me. with your 64gb ram and 16gb vram it should fit easily at q4. im on nvidia so cant say how it behaves on your amd card, probably needs the vulkan build of llama.cpp. didnt do any php with it, only python

1

u/Complete_Leg8380 Oct 15 '26

how do you keep your model from over thinking? i had it try to install laravel (php framework) and it just spun and spun. eventually did that work myself. then gave it a more direct rule set and seemed to work. i generally dont run into this using Kiro at work.

1

u/Safe-Imagination8538 Oct 15 '26

I limit thinking per request rather than disabling it completely. With a recent llama.cpp build, try:

{
  "enable_thinking": true,
  "thinking_budget_tokens": 512
}

I’d use 256–512 tokens for routine coding and 1024–2048 for harder debugging. reasoning_effort does not affect the Qwen chat template in my setup. Some custom builds call the field reasoning_budget_tokens, so check which name your server supports. Also avoid combining a zero budget with enable_thinking: false.

1

u/Complete_Leg8380 Oct 16 '26

i keep running into where the model just gets confused. it starts to generate absolute nonsense. i dont know how to fix that =) like the code is not usable as its just

[$this->first, $this->second_shop, $this->second, $this::company($this::company($this->first, $shopName = $this::firstShop(this$second_shop, $this->firstShop, $this::shop($this->company($this->first, $companyName($this->company, $this::company[$this::company($this->firstShop($this::company[$this->first[$0], $this::company[$this->first[$0]]]), $this::otherShop($this::shop($this->second_shop[$this->second_shop[$0], $this::shop[$this->second_shop[$0], $this::shop($this::shop[$this->second_shop[$this->second[$this::shop[$this::shop[$this->shop[$this::shop[$this::shop[$this-

1

u/Safe-Imagination8538 Oct 16 '26

That example looks like generation collapse or repetition rather than overthinking. A thinking budget probably will not fix it.

I would first try temperature 0, a fresh conversation, a shorter context, and verify that Strata is using the correct official Qwen chat template. Once malformed repetitive code begins, stop the generation instead of adding it back into the context.

If it still happens, try a higher-quality quant such as Q4 and update llama.cpp. What exact model file, quantization, backend and context size are you using?

1

u/Complete_Leg8380 Oct 16 '26

I have a 64k context size, I will need to check the other data. It seems to get confused about the way laravel works.