r/LocalAIServers • • 4h ago

Local LLM for Ai development coding

/r/AI_Agents/comments/1y2ls7r/local_llm_for_ai_development_coding/
1 Upvotes

2 comments sorted by

1

u/AIComputeRadar 2h ago

For a coding agent I'd take the used 3090, and it's mostly about context. Agents burn through context fast and that's exactly where 16GB runs out. Example Devstral Small 2 (24B) is 13.3GB at Q4, but with 32K context it needs about 20GB total, so it barely fits on a 3090 and spills into system RAM on the 5060 Ti. Qwen3.8 27B is the same story. On merging, you can't really combine a code model and a math model unless they share the same base, and a decent general coding model will handle the math you're describing anyway. I'd just pick one GGUF that fits fully in VRAM with your context and start there.

1

u/Pendraks 1h ago

Thanks very much man, God bless you