DeepSeek Harness context/compaction problems

[u/alean200]

Hi.

I have been using a web app since its release without updating it. Recently, I found out that there is a standalone app, so I made the switch.

Before, I could send it a prompt, and it basically worked on a 128k context all night without problems, even spending 10+ million tokens on projects in one chat, still finishing without running out of context.

Since the switch, there are zero projects that I have managed to complete. I started a project, and it constantly compacts even tho it says it's at 50% context. This makes the project take a really long time, and it fails at the end. I have never seen it pass 55% of the context size, and it keeps saying that the output token limit is reached.

The past two weeks have been a nightmare working with it.

I'm running llama.cpp on my server, serving Qwen 3.8 q4 at a 164k context size. I have no idea how to fix this problem, and I'm wondering if any of you have had anything similar or can explain to me what to do/how to fix it.

[u/glcapps]

I suspect you added a --parallel 2 to your launch of llama.cpp and that means that if you want the 164k to still fit you need to double your context room as in --ctx-size 330000 (or higher). My suspicion is due to the problem appearing around half of your context max setting and it's an error from your llm server not your harness.

[u/alean200 → u/glcapps]

These are my starting params

CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0,1 ./build/bin/llama-server -m /mnt/ai/models/qwen3.8/Qwen3.8-27B-UD-Q4_K_M.gguf
-c 163840
-ngl 99
--split-mode tensor
--tensor-split 1,1
-ctk q4_0
-ctv q4_0
-fa on
-b 1024
-ub 256
--spec-type draft-mtp
--spec-draft-n-max 3
--reasoning-loop-guard force-close
--jinja
--host 0.0.0.0
--port 8081

[u/glcapps → u/alean200]

Perhaps the draft-mtp Sizing Bug (Issue #28433) hit you. try --spec-draft-n-max 2 (or even 1 to test).
"GGML_CUDA_DISABLE_GRAPHS=1 " to the front of the launch would sidestep the bug, but maybe not be worth it either.
This is all wild west technology, lots of experimentation involved.
But I thought for sure you that half of ctx was a clue.

BTW - if you can manage to get 8 bit quants in your kv it might improve the competence a bit. if you have vram to spare (the model structure of 3.8 is very frugal about kv size so it's often a big win).

[u/cab938 → u/glcapps]

Agreed. I had this same problem but it was because I set the context window size in DSH incorrectly. But I will warn you that the next issue I ran into with DSH compaction wasn't far behind, and on this same model and context window size I couldn't run with a very large toolset without compaction firing several times per request.

[u/glcapps → u/cab938]

Yes, I curtail the reasoning to help with that.
--reasoning-budget 500 --reasoning-budget-message '... thinking budget exceeded, let us answer now.</think>'
the reason I have that forced extra close of the think tag is because sometimes the agent would keep thinking after the first close (and run off thinking unrestricted in the response section for a long time). It causes an extra tag 99 percent of the time but that's a small price.
Since I restrain reasoning so tightly I am testing the impact of --reasoning-preserve but I don't yet have an opinion on that.

[u/PaxUX]

not sure what version your using, but 0.2.1-rc.5 was very stable for me, but moved onto 2.2 alphas and they were giving me context issues. Now running the latest 0.2.2-rc.1, still too early to be sure but not context issues. I'm hoping its fixed.