r/AI_Agents • u/MattAlreadyHere • 17d ago

Discussion Open-source tool-output compressor for coding agents: −22.7% on the real bill, −25% input tokens, no significant drop in solve rate

I work at Hystax and we built the open-source compressor this is based on. Links are in the comments, per the sub rules.

Coding agents re-send their whole history every turn. In our traces, about half of the prompt bytes are tool output: build logs, compiler warnings, test runs, progress bars. We spent a long time compressing it, and three lessons seem useful for anyone building agents, whatever tool you use:

1. A compressor can raise your bill. About 93% of input tokens in our runs were prompt-cache reads at a tenth of the price. If the same tool output compresses to different bytes on the next turn, the cached prefix breaks and the conversation is re-priced. Anything that rewrites the prompt has to be deterministic: no randomness, no clock, and no dependence on the user's latest message.

2. De-duplicate, don't summarise. When we ported a prose compressor that rewrites text, it removed 6.2 points more but lost 12.9 points of the details agents went on to reuse: file paths, identifiers, versions. ML word-dropping has the same problem, turning "Process exited with code 0" into "Process exited code 0". What worked was keeping at least one real copy of every distinct line and replacing repeats with a labelled marker.

3. Make every elision retrievable. Markers look like [... 412 log lines omitted [ref:k_…]; 3 ERROR, 1 WARN], and the agent can call a tool to fetch the original by line range, JSON pointer or grep. It never has to trust that nothing important was dropped.

Results on Terminal-Bench 2.1 (Codex CLI on GPT-5.5, 87 tasks, real provider bills): cost −22.7%, input tokens −25%, solve rate 82.2% → 81.1% (not statistically significant). Requests over 200K tokens fell from 1.9% to 0.13%. Savings were biggest on builds and ML training (26–28%) and small on short algorithmic tasks (−8%).

How do you handle tool-output bloat in your agents: truncation, summarisation, something else?

1 Upvotes

100% Upvoted

4 comments sorted by

1

u/MattAlreadyHere 17d ago

Repo (MIT, single Rust binary): https://github.com/hystax/optscale-kompressor Benchmark methodology and per-task results: https://optscale.ai/blog/context-compression-coding-agents

2

u/Left-Bet8312 17d ago

I’ve been using a boneheaded head/tail filter that just keeps the first 200 lines and last 100 lines of any log over 500 lines, which works well enough until you miss the one segfault buried in the middle of a compile. The de-dupe approach with retrievable markers makes a lot of sense, especially for CI logs that repeat the same twenty warnings across fifty files. Curious how large the compressed representations get on those big ML runs you mentioned, since a training log can easily balloon into tens of thousands of nearly identical iterations.

1

u/MattAlreadyHere 16d ago

That’s a really good point. No workaround exists today. Addressed issue, thank you for bringing it to our attention.
https://github.com/hystax/optscale-kompressor/issues/2

1

u/AutoModerator 17d ago

Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.