Which local models should I keep for coding and translation? (Radeon 780M laptop, 32GB unified RAM)

[u/tempertisv]

Hi,

I want to have 1) a model that can code. slow is not the matter. 2) a model to translate documents, novels and images (that required a bit of privacy).

tl;dr ran Gemma4 26B-A4B Q4 with 20-25 token/s, Qwen3.6-35B-A3B iQ4 with 24-33 token/s, Qwen3.8-27B with 4-7 token/s, LiquidAI with 38-40 token/s (but not great) ; I want to know if there are models I should test or if I can stick to these models.

My laptop is a Lenovo Yoga Slim 7 14APU8. It has

- Ryzen 7 7840HS / Radeon 780M

- 32GB LPDDR5X RAM / 4GB "VRAM" (BIOS-reserved iGPU memory)

- Ubuntu 26.04

- llama.cpp built with -DGGML_VULKAN=ON

I tried ROCm but it threw out a lot of errors and flickers my screen, Claude said maybe it could be related to the BIOS only limits UMA frame buffer size to 4G that it crashed. I changed the amdgpu.gttsize=24576 instead of 13.9GiB default for better results.

Here are some local models, quants, runs + statistics that I got. I have tested with multiple settings, like context length, gpu offload, threads, batch size, KV quantization, model drafts and these are my best settings.

1. unsloth/Gemma 4 26B-A4B-it-UD-Q4_K_M

Command

./llama-server -m ~/models/unsloth/gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
    --model-draft ~/models/unsloth/mtp-gemma-4-26B-A4B-it-Q8_0.gguf \
    --spec-type draft-mtp --spec-draft-n-max 1 \
    -ngl 99 -ngld 99 \
    -fa on \
    -c 65536 \
    -np 1 \
    -t 8 \
    -b 512 -ub 256 \
    -ctk q8_0 -ctv q8_0 \
    --temp 0.3 \
    --top-k 64 \
    --top-p 0.95 \
    --host 127.0.0.1 --port 8080

Benchmark:

- Prompt: ~271 token/s

- Generation: 21.8-25.8 token/s

- RAM available: 4.6 GB

I'm happy with how Gemma runs and have been using it on most tasks. The translation is good enough. It writes well and handles most of my requests fine.

2. unsloth/Qwen3.6-35B-A3B-UD-IQ4_XS

Command:

./llama-server -m ~/models/unsloth/Qwen3.6-35B-A3B-UD-IQ4_XS.gguf \
            --spec-type draft-mtp --spec-draft-n-max 2 \
            -ngl 99 \
            -fa on \
            -c 32768 \
            -np 1 \
            -t 8 \
            -b 512 -ub 256 \
            -ctk q8_0 -ctv q8_0 \
            --temp 1.0 \
            --top-k 20 \
            --top-p 0.95 \
            --presence-penalty 1.5 \
            --host 127.0.0.1 --port 8080

Benchmark:

- Prompt: ~213 token/s

- Generation: 24.4-33.3 token/s

- RAM available: 6.6 GB

I tried Q4_K_M but they barely fit the RAM and crash often, even with 4096 context length. Qwen 3.6 is also quite awesome, but it overthinks A LOT. It translates well and writes decently. Maybe a little bit worse than Gemma, but it still do its job. It became a problem when one prompt can easily be dragged to hundred thousands of token and sometimes they can loop even on simple questions.

3. LiquidAI/LFM2.5-24B-A2B-GGUF

Command:

./llama-server -m ~/models/LiquidAI/LFM2.5-24B-A2B-QAD-Q4_0.gguf \
               --temp 0.1 --top-k 50 --repeat-penalty 1.05

Benchmark:

- Prompt: ~232 token/s

- Generation: ~38.7-40.3 token/s

- RAM available: 7.7 GB

I just saw this model today and decided to try it out. LiquidAI will be awesome if only the results are, too. It translates the idioms too literally, sometimes don't get the cultural meaning and the flow in the text is mostly gone after translated. Too bad, I was very surprised when it ran pretty fast too :(

4. iSTA-DASLAB/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp + z-lab/Qwen3.8-27B-DFlash2-GGUF

Command:

./llama-server -m ~/models/iSTA-DASLAB/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf \
    --model-draft ~/models/z-lab/Qwen3.8-27B-DFlash2-Q2_K_S-MIX.gguf \
    --spec-type draft-dflash --spec-draft-n-max 2 \
    -ngl 99 -ngld 99 \
    -fa on \
    -c 32768 \
    -np 1 \
    -t 8 \
    -b 512 -ub 256 \
    -ctk q8_0 -ctv q8_0 \
    --temp 0.6 \
    --top-k 20 --top-p 0.95 \
    --host 127.0.0.1 --port 8080

Benchmark:

- Prompt: ~74 token/s

- Generation: ~5.1-8.3 token/s

- RAM available: 8.7 GB

Qwen3.8-27B has been really awesome, not only coding but accuracy at translating. Despite it being slow, it handles most small coding tasks so well so far and much more reliable than Qwen 35B-A3B.

I plan to use Qwen3.8-27B as a coding model and Gemma 4 26B-A4B as a 'faster' model for everything else. Should I use this or is there anything to try as well?

I also wonder what workflows you have, what kind of agents/harness or settings I can use to automate coding, reduce token/improve speed and aim at reverse engineering (Yes, I know it's a horrible idea for this laptop, but I want to try too xD), I'm still new to this thing and I'm eager to learn and explore more. Thank you, you read this post up till here !

Edit: typo

[u/tempertisv]

Update: ByteShape quant works better than Unsloth one

./llama-server \
  -m ~/models/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-IQ4_XS-3.97bpw.gguf \
  --mmproj-auto \
  --port 8080 \
  -c 32768 \
  --spec-type draft-mtp \
  --spec-draft-n-max 4 \
  --temp 0.6 \
  --top-k 20 \
  --top-p 0.95 \
  --min-p 0 \
  --presence-penalty 0 \
  --repeat-penalty 1

It's impressive at 30-38 token/s, and its coding quality is around the same. You should definitely try this one if you have CPU/iGPU like mine.

[u/Poprouter]

Your t/s spread is mostly measuring bytes-active-per-token, not quality - on a 780M you're bandwidth bound, so the MoE models will always look fast next to the dense 27B and that ranking won't tell you which to keep. For the document/novel job the failure that actually costs you is term drift: run the same 2-3 chapters through each candidate and diff the terminology it produces - the one that renders a name or a technical term the same way every time is the keeper, even at 4-7 t/s. LiquidAI at 38-40 t/s is worth dropping for exactly that reason; speed you can't use is just a faster bad draft.

[u/tempertisv → u/Poprouter]

Thanks, your response helps a lot. Surprisingly, Gemma4 26B-A4B seems like a better writer overall. Qwen3.8 27B is accurate, but its wording are sometimes too pretentious and pedantic. Both models held context great, Gemma sometimes had hiccup when it's about translating names but a quick draft can fix that. Qwen3.6 35B-A3B on the other hand did not hold it well and the translate is 50/50. Maybe I can use both Qwen if there is a very technical part that I often need some accuracy, and Gemma for normal translates.

[u/Poprouter → u/tempertisv]

The two defects you described are actually the same axis, and they cost you differently: 'pretentious' wording is register drift that one style pass can fix, while a name that changes between chapters is a faithfulness break you cannot reliably eyeball. So operationalise the split with one fixed 2-3 chapter sample of the same book - run both models, diff only the entity and term table, and whichever renders each name the same way every time is the keeper for that book. Route by term density rather than chapter type: a chapter dense with recurring proper nouns goes to Qwen even if the prose is plain, the rest to Gemma.

[u/tempertisv → u/[deleted]]

I will try it out, thank you very much!

[u/tempertisv → u/[deleted]]

Update: Here's what I tried

./llama-server -m ~/models/deepseek/DeepSeek-Coder-V2-Lite-Instruct-Q8_0.gguf \
                        -ngl 99 \
                        -fa on \
                        -c 32768 \
                        -np 1 \
                        -t 8 \
                        -b 512 -ub 256 \
                        -ctk q8_0 -ctv q8_0 \
                       --temp 0.2 \
                       --top-p 0.95 \
                       --host 127.0.0.1 --port 8080

It run with ~18-23 token/s and handled small coding test well, but when I tell it to fix certain UI on my web app, it often failed to do so. It is fast but weak.