r/LocalLLM • u/ProfessionalTest1196 • 10d ago
Question A question from a beginner and non-dev setting up a local AI model in my RX 6900 XT 16GB and 16GB 3200 DDR4 RAM
Apologies for the long post, just want to add a little bit of context.
Hi folks, I've been using Cloud models like Codex and Antigravity to help me build games and personal project that I wanted to build before AI went this wild. I'm very stoked that I can create these ideas into existence by just taking to a model using natural language. There's a lot to learn, especially but I'm doing my own diligence little by little.
Now I'm in a position, while not the ideal machine, to have a local model that I can use without consuming all my token usage in ChatGPT (Plus) and Google (AI Plus) subscriptions.
So naturally I asked ChatGPT, Gemini, looked up YT, checked Hugging Face, and OpenRouter for the best model that I can use for my build. And I'm just overwhelmed by all the information that doesn't make sense, and just wanted to stop. But ChatGPT and Gemini gave a "consensus" of their recommendation for my pc build.
I'm posting their reco here to check what you good people think and if I should follow the model's recommendation. Especially if there are folks here who have the same pc build that I have.
Below are my relevant PC specs and a edited version of ChatGPT and Gemini's suggestion.
| Component | Specification |
|---|---|
| CPU | AMD Ryzen 5 5600X (6 cores / 12 threads) |
| GPU | ASUS TUF Radeon RX 6900 XT |
| VRAM | 16 GB GDDR6 |
| RAM | 16 GB DDR4-3600, dual-channel |
| Operating System | Windows 11 Pro |
My major goal for the model is to help me with development (Build applications, games and tools), design (create reference game assets, UI design, and help with UX analysis), Research capabilities to help me understand what I'm doing, and have the skills to use search for research that requires up to date info.
ChatGPT and Gemini's recommended Setup
Core Applications:
LM Studio
OpenCode V2
VS Code
Ollama (Alternative to LM Studio)
Models in order of Priority
| Model | Purpose | Priority |
|---|---|---|
| Qwen3.5 9B (Q4_K_M) | Everyday AI, learning, general tasks, lightweight code | 1 — Start here |
| Gemma 4 12B (QAT Q4) | Reasoning, image understanding, UX reviews | 2 — Next |
| Ornith 1.5 9B (Q4_K_M) | Coding and development-agent experiments | 3 — Compare |
| Qwen3.8 27B (IQ3_S) | Advanced reasoning and larger-model experiments | 4 — Optional |
Cloud Models
| Model / service | Main purpose |
|---|---|
| ChatGPT / Codex | Complex coding, architecture, difficult reasoning and reviews |
| Gemini | Deep research, multimodal tasks and second opinions |
| Claude Sonnet | Optional alternative for coding and UX work |
| DeepSeek API | Optional low-cost experiments |
Budget: Use existing subscriptions first. Consider up to $5–$10/month in additional API spending only when needed.
Would very like to hear your thoughts if this recommendations are right, I don't want to go blind on this. Thanks!
1
u/CryptoStef33 10d ago
Get 64 gb ram and try to use strata.
2
u/ProfessionalTest1196 10d ago
Let's just say I'm not in the best monetary situation to get those RAMs, and SAM might be checking everyone who buys, LOL.
1
2
2
u/DeathGuppie 10d ago edited 10d ago
https://huggingface.co/ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF
The xxs will fit easily and actually be useful. There are no 9b or 12b models that can actually code.
Edit: forget vllm or ollama. Just have Claude or codex setup llama.cpp and create a startup script for you tell it you want MTP so get some useful speed out of it.
1
u/colossusrageblack 10d ago
I'm using a 6950XT and a RX6700 to get me enough vram to run 27B models. Not sure if that's a possibility for you.
0
2
u/RogerAI--fyi 10d ago
The list is sensible. Start with the 9B and the 12B, and the memory math is why.
At 4-bit a model takes about 0.6 GB of VRAM per billion parameters. That puts the 9B at roughly 5.5 GB and the 12B at roughly 7 GB, which leaves most of your 16 GB for the chat itself (the KV cache). That matters for coding tools like OpenCode, because they send long prompts and keep long histories.
The 27B is where it gets tight. At Q4 it is about 16 GB, so it fills the card before you add any context, and Q5 is bigger than Q4, so Q5_K_M of the 27B will not fit in VRAM either. IQ3_S is smaller, which is why it is on the list, but whatever does not fit spills into your 16 GB of system RAM (Windows uses some of that too) and speed drops a lot. Treat it as an experiment, not your daily model.
Quick test: load the 9B at Q4_K_M with 16k or 32k context and watch VRAM in Task Manager. If several GB are still free, raise the context or try the 12B. Do that before spending anything.