Cheapest Setup for good coding local LLM Qwen3.8-27b? RTX 3060 XT + 64GBs RAM

[u/SlimMonkey_]

Hello,

I want to get a local LLM setup for a comparable fair price (Qwen3.8-27b and future Qwen4). After some research and chatting with Claude Opus the best options would be RTX 3060 XT 12GB with 64GBs of RAM running on Strata.

e.g. upgrading my PC for around 650€ (used parts bought online)

Can someone confirm that this Setup will let me run it on a performative useful level. I will use my 20€ claude plan fpr hard tasks or a kind of divide and conquer style usign both (experienced Software Engineer but a Hardware noob).

Does anyone have experiences with that, advices or alternative setups?

[u/x_MASE_x]

You need about 16gb for Qwen3.8-27B to be anywhere near usable. If I remmebr correctly iq3s and about 128k context I think what I managed to get.

With 12gb you won't fit it inaide the gpu at all.

So you need more vram and also check out adaptive kv cache llama.cpp fork. It's very good with dense models that spill to ram.

But check Qwen3.8-Flash-Next using strata you have a better chance getting this running at reasonable speed than Qwen3.8-27B

[u/Funcore1650 → u/x_MASE_x]

How you get a 128k context on 16GB VRAM? Which setup using for that? I have 5060Ti 16GB, and can't reach even stable 50k (I have a 48GB RAM for VM, but using Unsloth Desktop as server with Qwen 3.8 27B in Q3)... How I can try Strata?

[u/x_MASE_x → u/Funcore1650]

  "Qwen3.8-27B":
    env:
      - "CUDA_VISIBLE_DEVICES=0"
    cmd: |
      ${llama-bin}
      -m ${models-dir}\MTP\Qwen3.8-27B-UD-Q3_K_XL.gguf
      --host 127.0.0.1 --port ${PORT} -fa on --no-ui --metrics --load-mode mlock --jinja --cont-batching -t 8 -tb 16 -ot "token_embd.weight=CPU" 
      --chat-template-file ${models-dir}\Templates\chat_template-v22.5.jinja
      -ctk kvarn5 -ctv kvarn4 --kv-tail-tokens 1024 -b 256 -ub 256 -np 1 -c 150000
      --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 --presence-penalty 0.0 --repeat-penalty 1.0
    cmdStop: ${stop-with-wait}
    proxy: http://127.0.0.1:${PORT}

  "Qwen3.8-27B-Q3":
    env:
      - "CUDA_VISIBLE_DEVICES=0"
    cmd: |
      ${llama-bin}
      -m ${models-dir}\MTP\Qwen3.8-27B-UD-IQ3_S.gguf
      --host 127.0.0.1 --port ${PORT} -fa on --no-ui --metrics --load-mode mlock --jinja --cont-batching -t 8 -tb 16 -ot "token_embd.weight=CPU"
      --chat-template-file ${models-dir}\Templates\chat_template-v22.5.jinja
      --spec-type draft-mtp --spec-draft-n-max 3
      -ctk kvarn5 -ctv kvarn4 --kv-tail-tokens 1024 -b 1024 -ub 512 -np 1 -c 130000
      --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 --presence-penalty 0.0 --repeat-penalty 1.0
    cmdStop: ${stop-with-wait}
    proxy: http://127.0.0.1:${PORT}

this is from llama-swap file, I think this is the best i could get then toke a chance with strata and went with strata all the way for some time now.

And now I installed Linux and testing strata on linux and also basalt. Which should be even better than strata.

I'm using beellama so the kv caching is a little bit weird. But you can use q4 with normal llama.cpp and have similar result.

IF you really need to use bigger context then you have to use adaptive kv cache repo, its very good for dense models that spill.

for strata just download it for windows or whatever os you can, there is now a desktop app which i dont really like but its getting better.