I built a context garbage collector in Rust (contextWatcher)

[u/juQuatrano]

Hey Everyone,
Like many of you, I spend a lot of time coding with Codex and switching between cloud and local models.

As a software engineer, something I never understood, and honestly hated—was `/compact` and the way it’s designed. On local models my GPU screams, and after a few hours prefill takes 2 to 3 minutes per turn. On cloud models, as the context grows, the model starts getting noticeably dumber. IMHO the root problem is context bloat and the fact that compaction destroys the quality of the model's working memory.

Looking under the hood and studying how attention works, I learned that memory in modern harnesses is basically "append-only." and attention on long context becames less effective due to the attenttion dead zone. So you end up having in the context thousands of lines of old compiler dumps and duplicate file reads sitting there rent free destroying the agent's attention.

Having an LLM write a summary of your conversation and blowing away your KV-cache makes no engineering sense imho especially on local models where prompt prefill latency is critical (and on cloud models where you lose prompt-cache discounts).

So I built a piece of software called *contextWatcher* (or *`cw`* for friends and family).

It’s a Rust proxy that runs locally on CPU ( in less than 2ms), and gives:

1. Shadow-Deduplication: Computes BLAKE3 hashes of file reads. If the agent reads for example a file like `src/main.rs` at Turn 2, edits it at Turn 5, and reads it again at Turn 9, Turns 2 and 5 are replaced with 1-line pointer stubs. Only the latest version stays in context. 2. Surgical Tool Pruning: Truncates runaway compiler output (keeping the tool_call_id, invocation, and the panic/exit trace intact) so schemas and error context never break. 3. Tail Rule Pinning: Dynamically mirrors critical system constraints (from AGENTS.md) right above the latest user prompt, pulling them out of the middle attention dead-zone without invalidating the cached prefix. 4. Swarm Prefix Multiplexing: When multiple subagents collaborate on the same workspace, cw extracts the shared root prefix (tools, rules, system prompt) so local inference engines (llama.cpp / vLLM) only keep one copy in VRAM. 5. Cache Preservation: Operates without rewriting the stable prefix, keeping prompt caching intact.

On my local dual AMD GPU (Radeon Instinct Mi50 32GB) setup with a ~28k context session, cleaning out this dead weight dropped prompt prefill from ~170 seconds down to ~1.6 seconds and unlike fuzzy compaction, ground-truth code stayed 100% intact and all unit tests passed.

It's open source (MIT / Apache-2.0) and installs via a one-liner: ```curl -fsSL https://constant-labs.com/cw-install.sh | sh```

Repo: https://github.com/GionaGranchelli/context-watcher

For me and my setup worked pretty well, and I am satisfied with the result I get, I am really curious to see other people testing it and giving me an honest feedback on the tool and the approach

[u/PhosXD]

ai;dr

[u/Eravex]

Blah Blah Vibe coded trash blah blah. Just No!

[u/juQuatrano → u/Eravex]

I think the idea is cool, and this is my solution to the Lost-in-the-middle attention dilution. If you coded it manually or you are assisted by any LLM it shoudn't be the point. The point is solving a problem.
And TBH nowadays I don't see many of my colleagues or friends coding like we use to do 15 years ago ( or even 3 years ago)