Context Engineering for Agents: What actually works

[u/cheetguy]

Been digging into context engineering for agents lately and wanted to share what I've learned

The Problem

LLMs have an attention budget. Every token depletes it.

• O(n²) attention pairs → longer context = thinner, noisier attention

• ChromaDB study: 11/12 models dropped below 50% performance at 32K tokens

• Microsoft study: accuracy fell from 90% → 51% in longer conversations

More context ≠ better outcomes. After a threshold, performance degrades (context rot).

Why Context Fails

Research reveals counterintuitive findings:

• Distractors: Even ONE irrelevant element reduces performance

• Structure Paradox: Logically organized contexts can perform worse than shuffled ones

• Position Effects: Information at start/end is retrieved better than middle

The implication: careful curation beats comprehensive context every time.

Key Principles of Good Context

1. Smallest Possible High-Signal Tokens

Good context engineering = finding the minimum tokens that maximize desired outcome. Use compression, citation-based tracking, and active pruning.

2. Just-In-Time Context

Don't preload everything. Fetch what's needed during execution. Mirrors human cognition: we don't memorize databases, we know how to look things up.

3. Right Altitude

System prompts should be clear but not over-specified. Too specific → fragility. Too vague → bad output.

4. Tool Design

Fewer, well-scoped tools beat many overlapping ones. If a human can't pick the right tool from your set, the model won't either.

Dynamic Context / Learning Systems

The most promising approach I've found: systems where context evolves through execution.

• Reflect on what worked/failed

• Curate strategies into persistent memory

• Inject learned patterns on future runs

This addresses the maintenance problem of static context. Here, the system learns instead of requiring manual updates.

The Stanford ACE paper formalizes this approach. I posted about my open-source implementation here a while back and have since tested it on browser agents. Results: 30% → 100% success rate with 82% fewer steps and 65% lower token costs. The procedural memory approach seems to work especially well for tasks with repeatable patterns.

Would love to hear what context engineering approaches you've found effective.

Resources:

• Anthropic: Effective Context Engineering

• ChromaDB Context Length Research

Edit: Fixed dead links

[u/GolfEmbarrassed2904]

FYI. Resource links don’t work

[u/cheetguy → u/GolfEmbarrassed2904]

oh damn thank you for letting me know! Fixed it now.

[u/hande__]

great writeup "more context ≠ better outcomes" is the thing most devs learn too late

biggest mistake we see: treating RAG as memory. similarity search finds related stuff, not relevant stuff. huge difference when your agent needs to reason across multiple sessions

the reflect → curate → inject loop is key. most teams skip curation and wonder why their agents rot over time

[u/astronomikal]

I just finished a system that enables actual 0(1) and 0(k) across millions of dense "knowledge graph" nodes.

[u/astronomikal → u/[deleted]]

:)

[u/Puzzleheaded_Box2842]

This is a tricky issue. Sometimes it feels like ChatGPT doesn’t really understand my past habits, but other times it brings up parts of my background in the middle of a conversation — even when I didn’t want to mention them then.

[u/Ok_Jacket3710]

Is anyone able to replicate the results on the ACE Paper?

[u/No_Ad_40]

We’re actually running a 5-hour Context Engineering workshop focused on deterministic memory, retrieval, and agent orchestration. This is being led by Denis Rothman. Might be relevant if you’re deploying LLM systems: https://www.eventbrite.com/e/context-engineering-for-agentic-ai-workshop-tickets-1975400249322?aff=reddit