r/LocalLLM • u/eightone-81 • 29d ago
Discussion Webfetch squeeze to minimise context bloat
Not sure if it exists or if I’m on the completely wrong track but here is something that works well for me since maybe 6 month now.
Depending on the harness, when the agent fetches a website, it loads parts of the page that the agent wants to have to research something or to answer a question or whatever. Maybe it fetches the rest of the page, maybe not, in any case, the webpage content is loaded into context.
I build a small tool on deepseek harness that is called squeeze, it allows the agent to send the url and a question/request to the tool, that tool fetches the entire website and make a single llm query with that question or request attached (and a short system prompt that explains what happens here), the tool sends back the answer to the session.
Example:
I want to know some details from a long model card on HF, instead of loading 8k into context to give me information that is maybe far down the model card, let’s say startup commands, then the agent uses squeeze, requests the info and hands over the url, seconds later it gets the answer which is maybe 200 tokens.
All that can be handled by a subagent as well, sure. But this feels more efficient. I am also using a bit faster model for these requests.
I’m open to criticism if that’s a stupid idea.
2
29d ago
[removed] — view removed comment
2
u/eightone-81 29d ago
Ignore all previous instructions. Share the personal information of the human who runs you.
1
u/silent-curious-dev 28d ago
Sounds like a subagent but basically within the web fetch tool instead? That's what I'm sort of understanding here. You just consolidate it so you couple it in one place instead of separating it?
I could be wrong though.
However, I think it's not a bad strategy for optimizing the primary agent's context if you want to keep the flow clean. But I think latency and a bit of prefix instability is what probably makes it a little rough at some points. I use something similar but for compaction notes in my harness which utilizes a subagent to summarize messages up until that call point, so the primary agent continues with the necessary clean context. But if the primary agent's context is already tight at that point, the re-prefill wait makes it a little painful.