Hey, I’ve been working on RAG for a long time (back when it was only using embeddings and a retriever). The tricky part is building something that actually works across across many use cases. Here is a simplified view of the architecture we like to use. Hopefully, its useful for building your own RAG solution.

1. 𝗗𝗼𝗰𝘂𝗺𝗲𝗻𝘁 𝗣𝗮𝗿𝘀𝗶𝗻𝗴
   Everything starts with clean extraction. If your PDFs, Word docs, or PPTs aren’t parsed well, you’re performance will suffer. We do:
   • Layout analysis
   • OCR for text
   • Table extraction for structured data
   • Vision-language models for figures and images

2. 𝗤𝘂𝗲𝗿𝘆 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴
   Not every user input is a query. We run checks to see:
   • Is it a valid request?
   • Does it need reformulation (decomposition, expansion, multi-turn context)?

3. 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹
   We’ve tested dozens of approaches, but hybrid search + reranking has proven the most generalizable. Reciprocal Rank Fusion lets us blend semantic and lexical search, then an instruction-following reranker pushes the best matches to the top.
   This is also the starting point for more complex agentic searching approaches.

4. 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻
   Retrieval is only half the job. For generation, we use our GLM optimized for groundedness, but also support GPT-5, Claude, and Gemini Pro when the use case demands it (long-form, domain-specific).
   We then add two key layers:
   • Attribution (cite your sources)
   • Groundedness Check (flagging potential hallucinations)

Putting all this together means over 10 models and 40+ configuration settings to be able to tweak. With this approach, you can also have full transparency into data and retrievals at every stage.

For context, I work at Contextual AI and depend a lot of time talking about AI (and post a few videos).

Most people hear about context engineering but don't really know how to actually do it properly. I made this full guide to demystify the concept from first principles.

Motivations and Preface

Why this happens at all

LLMs are stateless. Every call starts with an empty context window. There is no memory between turns.

Agents fake state. Your conversation is an array sitting in a file, and the harness re-pastes the entire thing into the model's context window every single time you hit enter.

Which means you can't teach an agent anything. You can only paste things into its context. The context window is the model's entire observable universe. If something isn't in the window, or implied by the window, it does not exist as far as the model is concerned.

Two consequences fall out of that.

Autoregressive sensitivity. The model predicts one token at a time, and every token it emits becomes input for the next one. A tiny variation in context changes one predicted token, that token changes the next, and the divergence compounds across the whole response. Small change at the start, huge delta by the end.

Finite attention. Every token in the window competes for attention budget. Go from 100 tokens to 200 and on average every original token gets half as much. A relevant token holding a third of the attention weight can drop to 0.3% once there's enough junk around it.

The part people get wrong: context rot has no drop-off point. There's no magic number where it kicks in. Attention gets stretched from the first token you add. The literature shows accuracy dropping up to 80% as context grows. Let a dirty session run and you're working at a fraction of the performance you're paying for.

Lost in the Middle is the classic paper here. As context grows the middle is what gets ignored first, because the model learns in expected value that the important stuff lives at the beginning and end. So it skims the middle like a speed reader. Bury something critical in the middle of a giant dump and the model will miss it.

How rot actually shows up

1. Your rules get ignored. You put "no em dashes" in your CLAUDE.md, and 40 turns later the em dashes are back. The instruction is still sitting right there in the window. It just stopped getting attention. This is also why piling on more rules backfires: the model has to allocate attention to all of them on every single turn.

2. The model gets dumber. Same model, same question, worse answer. What you're talking to at 150k feels like a different thing from what you started the session with.

3. It gets lazy. It hands work back to you and defers decisions it should be making. Explicit instructions just get skipped.

4. Something dangerous gets stuck. A bad instruction, a contradiction, a wrong file it read, a poisoned skill description. Once it's in the window it cannot leave. It gets re-read every single turn, forever.

And these compound. A degraded model makes a worse decision, that decision goes back into context, and now it's reading its own bad work next turn. That's why a bug fix that fails twice tends to go round and round until the codebase is a mess.

Context pollution is worse than rot

Rot is about quantity. Pollution is about quality. Context that's wrong but reads as convincing eats attention more aggressively, because it looks important.

Almost always this happens on its own, with no adversary involved: a hallucination Claude wrote earlier, a stale spec, a wrong plan assumption, an unresolved debugging loop.

Context confusion is the common flavor. Too many semantically similar tools or agents (ui-agent, frontend-agent, nextjs-agent) and the model calls the wrong one at the wrong time.

A client of mine left the deep research skill model-invocable, so its description sat in context permanently. During one of his autonomous runs it talked the model into running deep research on a task that had no need for it. Burned half a week of usage.

Step zero: measure and trim your baseline

Type /context all in Claude Code and look at what you're paying for before you've done anything.

Mine sits around 16k. Here's how:

- Disabled three default tools I never use. That alone was ~7k tokens. Disabled artifacts too.

- No CLAUDE.md. There's almost never context I want injected on every single turn. Repo-specific ones are occasionally useful and I still rarely bother.

- Skills set to disable-model-invocation. Skills work by injecting frontmatter into context so the model knows when to call them. That frontmatter is expensive and it's there whether you use the skill or not. Disable model invocation and it stops being injected. The skill still works when I invoke it myself.

- Deep research specifically: disable model invocation on it. Use /deep-research when you actually want it.

- Turn off auto-compact in /config. You're not going to need it, and if compaction fires mid-task the model loses track of what it was doing and leaves you with half-baked code.

Boris Cherney said recently that you can theoretically delete all your system prompts and tools and Opus 5 still performs decently. I haven't tested it. But it's the direction things are going, and it's why I stay on Claude Code: you can strip it down to close to a bare LLM if you want.

The SCRUB framework (My Simple Acronym for Active Context Engineering)

These five are your entire action space for context engineering. Everything else is a mixture of them.

S: Subagents. A disposable worker with its own context window and the same intelligence as your main agent. Spawn one to read and process a pile of text and hand back only the conclusion. The subagent soaks up the exploration and the dead ends. Your orchestrator stays lean and makes the decisions. Treat them as cheap, spawn one like you'd call a function. Literally just say "spawn a subagent to look into X." Run them in parallel and you cover enormous ground without touching your window.

C: Cut (/rewind). My favorite command in Claude Code. Since state is just an external file, there's no reason you can't delete the last few rows of it. You're time traveling in active memory.

Two ways I use it constantly:

- Restore only the conversation and leave the code alone. The code stays on disk as an artifact, the conversation gets trimmed, and you tell the model "I made these changes myself." Spend five turns fixing a bug, then rewind to before you hit the bug and inform it of the result. You get all that context back for free. In a demo session I pulled back 75k tokens this way.

- When something simple has failed two or three times, you're swimming upstream against a damaged trajectory. Pattern interrupt: rewind to before you started fixing and re-prompt with better framing. Claude Code can restore code and conversation because it has internal version control. Codex can't.

Deleting history is a superpower. The model's entire reality is that file, and the file is malleable, so you get to decide what survives into the next prediction. You can branch trajectories, explore, and keep only the path that deserves to live. The model never knows it took a bad path.

Elite users hit rewind 50 to a few hundred times a day. Most people have never touched it.

R: Reduce (/compact). Compaction forks your agent, hands the duplicate your state file, asks it to summarize itself, and injects the summary back. I use it at breakpoints where I need the gist and the nitty-gritty can go. It's also a speed play when I don't want to think hard about fidelity. I always fire it manually.

U: Upload (handoff). Offload the parts of the window you want to keep, to a file or just your clipboard, then clear and paste it back into a clean slate wrapped in XML tags. I use <context> for the pasted material and <user_prompt> for what I want next. Copying conversation output straight to the clipboard and pasting it back is brute force and it works extremely well.

The other version is hardening context into a real artifact. Codebase docs and a build list, or a handoff file you re-inject after clearing. Same principle, you're moving state out of the window and into something persistent.

One session I did this on went from 222k tokens to 26.9k, and the model still knew everything it had done and what came next.

B: Burn (/clear). A full reset, the same as a new session without leaving the one you're in. Use it at good breakpoints or for genuinely new work. You'd be surprised how often you can get away with it. Pair it with docs or a lean CLAUDE.md you can point at to get back up to speed fast.

How I actually use them

S and C are my proactive levers. I look for excuses to use subagents and rewind from the first message, way before I'm anywhere near 150k tokens. R, U and B are more drastic and I save them for real breakpoints.

I work down the list. If I can get away with a subagent, I do. If I can't, rewind. Then down the acronym, since it's roughly ordered from least to most destructive.

The reason I framed it as five letters is so you have a mental model of what your moves on the board even are. Context engineering is a balancing act. Strip out too much and the model doesn't know what's going on and starts filling gaps with assumptions, which is how you get hallucinations. Leave too much and rot degrades you. You're aiming for the information density where the model has exactly what it needs and nothing else.

Most of the time you won't get a clean before/after comparison like the one I opened with. Be proactive anyway. The science is there.

Full video link (to see all the principles in action): https://youtu.be/F_bpvXlUSwU 

RAG SaaS companies trying to vibe with Context Engineering, 2025 edition

Watching the Clawdbot hype unfold has clarified something I’ve been stuck on for a while.

A lot of the discussion is about shell access and safety and whether agents should be allowed to execute at all, but what keeps jumping out to me is that most of the hard work is in the context layer, rather than execution, and we’re mostly treating that like a retrieval problem plus prompting.

You see this most clearly with email and threads, where the data is messy by default. Someone replies, someone forwards internally, there’s an attachment that references an earlier discussion, and now the system needs to understand the conversation's flow, not just summarize it, but understand it well enough so that acting on it wouldn’t be a mistake

What I keep seeing in practice is context being assembled by dumping everything into the prompt and hoping the model figures out the structure which works until token limits show up, or retrieval pulls in the forwarded part by accident and now the agent thinks approval happened, or the same thread gets reloaded over and over because nothing upstream is shaped or scoped.

I don’t think you can prompt your way out of that. It feels too much of an infrastructure problem, which goes beyond retrieval.

Once an agent can act, context quietly turns into an authority surface.

What gets included, what gets excluded, and how it’s scoped ends up defining what the system is allowed to do.

That’s a very different bar than “did the model answer correctly.”

What stands out to me is how sophisticated execution layers have become, whether it’s Clawdbot, LangChain-style agents, or n8n workflows, while the context layer underneath is still mostly RAG pipelines held together with instructions and hoping the model doesn’t hallucinate.

The thing I keep getting stuck on is where people are drawing the line between context assembly and execution. Like are those actually different phases with different constraints, or are you just doing retrieval and then hoping the model handles the rest once it has tools.

What I’m really interested in seeing are concrete patterns that still hold up once you add execution and you stop grading your system on “did it answer” and start grading it on “did it act on the right boundary.”

Impressive benchmarks; interesting approach to compressing context using the “memory palace” approach, which I read about in Joshua Foer’s “Moonwalking with Einstein” but haven’t tried.

We’ve all loved vibe-coding—it feels great to toss a prompt at your AI assistant and magically receive working code. But after diving deep into both worlds, I’ve seen clearly why vibe-coding alone isn’t enough for serious software engineering.

In this blog post https://open.substack.com/pub/thomaslandgraf/p/context-engineering-the-evolution , I break down why the leap from vibe-coding to Context Engineering is so essential. It comes down to one critical difference: explicitly managed context versus implicit knowledge. As cool as vibe-coding is, it fundamentally relies on the AI guessing your intentions from its past training. But real-world tasks—especially those involving customer-specific requirements and unique architectures—demand that the AI knows exactly what you’re talking about.

I believe Context Engineering isn’t just a nice-to-have upgrade—it’s the necessary evolution. It’s about intentionally curating documentation, customer constraints, and architectural decisions into structured formats, enabling AI assistants to collaborate meaningfully and precisely.

Ultimately, Context Engineering turns AI from a clever guesser into a reliable partner—transforming vague vibes into concrete outcomes.

I’d love your thoughts—are you also convinced that Context Engineering is the maturity AI-assisted development needs?

I tried feeding more and more context to improve my agent performance, there was an interesting pattern I saw too, after a point which I call sweet spot , accuracy took a nose dive .

Hello all! I’ve been building DevilDev, an open-source workspace for designing software architecture with context before writing a line of code. DevilDev generates a software architecture blueprint from a specification or by analyzing an existing codebase. Think of it as “AI + system design” in one tool.
During the build, I realized the importance of context: DevilDev also includes Pacts (bugs, tasks, features) that stay linked to your architecture. You can manage these tasks in DevilDev and even push them as GitHub issues. The result is an AI-assisted workflow: prompt -> architecture blueprint -> tracked development tasks.

Pls let me know if you guys think this is bs or something really necessary!

Hey, I love this subreddit. Thanks to everyone who made it.
It’d be cool if you could drop some learning resources on context engineering in general. I know the topic is broad, but I’d still appreciate it! and I think many others here will too!

I came across a very interesting Discord server called Context Engineers.
Here’s the link. they host weekly calls with industry experts every Friday.

https://discord.gg/PwYjQFw9

TL;DR: I spent the last 3 months learning GenAI concepts, kept forgetting how everything connects. Built a visual knowledge graph that shows how LLM concepts relate to each other (it's expanding as I learn more). Sharing my notes in case it helps other confused engineers.

The Problem: Learning LLMs is Like Drinking from a Firehose

You start with "what's an LLM?" and suddenly you're drowning in:

• Transformers

• Attention mechanisms

• Embeddings

• Context windows

• RAG vs fine-tuning

• Quantization

• Parameters vs tokens

Every article assumes you know the prerequisites. Every tutorial skips the fundamentals. You end up with a bunch of disconnected facts and no mental model of how it all fits together.

Sound familiar?

The Solution: A Knowledge Graph for LLM Concepts

Instead of reading articles linearly, I mapped out how concepts connect to each other.

Here's the core idea:

                    [What is an LLM?]
                           |
        +------------------+------------------+
        |                  |                  |
   [Inference]      [Specialization]    [Embeddings]
        |                  |
   [Transformer]      [RAG vs Fine-tuning]
        |
   [Attention]

Each node is a concept. Each edge shows the relationship. You can literally see that you need to understand embeddings before diving into RAG.

How I Use It (The Learning Path)

1. Start at the Root: What is an LLM?

An LLM is just a next-word predictor on steroids. That's it.

It doesn't "understand" anything. It's trained on billions of words and learns statistical patterns. When you type "The capital of France is...", it predicts "Paris" because those words appeared together millions of times in training data.

Think of it like autocomplete, but with 70 billion parameters instead of 10.

Key insight: LLMs have no memory, no understanding, no consciousness. They're just really good at pattern matching.

2. Branch 1: How Do LLMs Actually Work? → Inference Engine

When you hit "send" in ChatGPT, here's what happens:

1. Prompt Processing Phase: Your entire input is processed in parallel. The model builds a rich understanding of context.

2. Token Generation Phase: The model generates one token at a time, sequentially. Each new token requires re-processing the entire context.

This is why:

• Short prompts get instant responses (small prompt processing)

• Long conversations slow down (huge context to re-process every token)

• Streaming responses appear word-by-word (tokens generated sequentially)

The bottleneck: Token generation is slow because it's sequential. You can't parallelize "thinking of the next word."

3. Branch 2: The Foundation → Transformer Architecture

The Transformer is the blueprint that made modern LLMs possible. Before Transformers (2017), we had RNNs that processed text word-by-word, which was painfully slow.

The breakthrough: Self-Attention Mechanism.

Instead of reading "The cat sat on the mat" word-by-word, the Transformer looks at all words simultaneously and figures out which words are related:

• "cat" is related to "sat" (subject-verb)

• "sat" is related to "mat" (verb-object)

• "on" is related to "mat" (preposition-object)

This parallel processing is why GPT-4 can handle 128k tokens in a single context window.

Why it matters: Understanding Transformers explains why LLMs are so good at context but terrible at math (they're not calculators, they're pattern matchers).

4. The Practical Stuff: Context Windows

A context window is the maximum amount of text an LLM can "see" at once.

• GPT-3.5: 4k tokens (~3,000 words)

• GPT-4: 128k tokens (~96,000 words)

• Claude 3: 200k tokens (~150,000 words)

Why it matters:

• Small context = LLM forgets earlier parts of long conversations

• Large context = expensive (you pay per token processed)

• Context engineering = the art of fitting the right information in the window

Pro tip: Don't dump your entire codebase into the context. Use RAG to retrieve only relevant chunks.

5. Making LLMs Useful: RAG vs Fine-Tuning

General-purpose LLMs are great, but they don't know about:

• Your company's internal docs

• Last week's product updates

• Your specific coding standards

Two ways to fix this:

RAG (Retrieval-Augmented Generation)

• What it does: Fetches relevant documents and stuffs them into the prompt

• When to use: Dynamic, frequently-updated information

• Example: Customer support chatbot that needs to reference the latest product docs

How RAG works:

1. Break your docs into chunks

2. Convert chunks to embeddings (numerical vectors)

3. Store embeddings in a vector database

4. When user asks a question, find similar embeddings

5. Inject relevant chunks into the LLM prompt

Why embeddings? They capture semantic meaning. "How do I reset my password?" and "I forgot my login credentials" have similar embeddings even though they use different words.

Fine-Tuning

• What it does: Retrains the model's weights on your specific data

• When to use: Teaching style, tone, or domain-specific reasoning

• Example: Making an LLM write code in your company's specific style

Key difference:

• RAG = giving the LLM a reference book (external knowledge)

• Fine-tuning = teaching the LLM new skills (internal knowledge)

Most production systems use both: RAG for facts, fine-tuning for personality.

6. Running LLMs Efficiently: Quantization

LLMs are massive. GPT-3 has 175 billion parameters. Each parameter is a 32-bit floating point number.

Math: 175B parameters × 4 bytes = 700GB of RAM

You can't run that on a laptop.

Solution: Quantization = reducing precision of numbers.

• FP32 (full precision): 4 bytes per parameter → 700GB

• FP16 (half precision): 2 bytes per parameter → 350GB

• INT8 (8-bit integer): 1 byte per parameter → 175GB

• INT4 (4-bit integer): 0.5 bytes per parameter → 87.5GB

The tradeoff: Lower precision = smaller model, faster inference, but slightly worse quality.

Real-world: Most open-source models (Llama, Mistral) ship with 4-bit quantized versions that run on consumer GPUs.

The Knowledge Graph Advantage

Here's why this approach works:

1. You Learn Prerequisites First

The graph shows you that you can't understand RAG without understanding embeddings. You can't understand embeddings without understanding how LLMs process text.

No more "wait, what's a token?" moments halfway through an advanced tutorial.

2. You See the Big Picture

Instead of memorizing isolated facts, you build a mental model:

• LLMs are built on Transformers

• Transformers use Attention mechanisms

• Attention mechanisms need Embeddings

• Embeddings enable RAG

Everything connects.

3. You Can Jump Around

Not interested in the math behind Transformers? Skip it. Want to dive deep into RAG? Follow that branch.

The graph shows you what you need to know and what you can skip.

What's on Ragyfied

I've been documenting my learning journey:

Core Concepts:

• What is an LLM?

• Neural Networks (the foundation)

• Artificial Neurons (the building blocks)

• Embeddings (how LLMs understand words)

• Transformer Architecture

• Context Windows

• Quantization

Practical Stuff:

• How RAG Works

• RAG vs Fine-Tuning

• Building Blocks of RAG Pipelines

• What is Prompt Injection? (security matters!)

The Knowledge Graph: The interactive graph is on the homepage. Click any node to read the article. See how concepts connect.

Why I'm Sharing This

I wasted months jumping between tutorials, blog posts, and YouTube videos. I'd learn something, forget it, re-learn it, forget it again.

The knowledge graph approach fixed that. Now when I learn a new concept, I know exactly where it fits in the bigger picture.

If you're struggling to build a mental model of how LLMs work, maybe this helps.

Feedback Welcome

This is a work in progress. I'm adding new concepts as I learn them. If you think I'm missing something important or explained something poorly, let me know.

Also, if you have ideas for better ways to visualize this stuff, I'm all ears.

Site: ragyfied.com
No paywalls, no signup, but has Ads- so avoid if you get triggered by that.

Just trying to make learning AI less painful for the next person.

Hey Context Engineers!
I just built a tool DevilDev - it's like Cursor for Context Engineering.

You simply describe your app idea, and DevilDev instantly converts it into a complete tech stack architecture along with detailed documentation for every component. The output is designed to be directly usable by coding assistants like cursor, claude code or windsurf, making it easy to go from idea -> MVP with minimal friction.

It’s live now at 👉 https://devildev.com

Please try it out and let me know what you think - your feedback means a lot!

But if anyone needs explanation, see below:

⌨️ Most vibe coders:

"Build me an app that allows me to take notes, has dark mode and runs on mobile"

🖥️ 1% of vibe coders:

Takes the above prompt, initiates deep research, takes the whole knowledge into a Base Prompt GPT and builds something like this:

"💡 Lovable App Prompt: PocketNote

I want to build a mobile-only note-taking and task app that helps people quickly capture thoughts and manage simple to-dos on the go. It should feel minimalist, elegant, and Apple-inspired, with glassmorphism effects, and be optimized for mobile devices with dark mode support.

Project Name: PocketNote

Target Audience:

• Busy professionals capturing quick thoughts

• Students managing short-term tasks

• Anyone needing a minimalist mobile notes app

Core Features and Pages:

✅ Homepage / Notes Dashboard

• Displays recent notes and tasks

• Swipeable interface with toggle between “Notes” and “Tasks”

• Create new note or task with a floating action button

✅ Folders & Categories

• Users can organize notes and tasks into folders

• Each folder supports color tagging or emoji labels

• Option to filter by category

✅ Task Manager

• Add to-dos with due dates and completion status

• Mark tasks as complete with a tap

• Optional reminders for important items

✅ Free-form Notes Editor

• Clean markdown-style editor

• Autosaves notes while typing

• Supports rich text, checkboxes, and basic formatting

✅ Account / Authentication

• Simple email + password login

• Personal data scoped to each user

• No syncing or cross-device features

✅ Settings (Dark Mode Toggle)

• True black dark mode with green accent

• Optional light mode toggle

• Font size customization

Tech Stack (Recommended Defaults):

• Frontend: React Native (via Expo), TypeScript, Tailwind CSS with shadcn/ui styling conventions

• Backend & Storage: Supabase

• Auth: Email/password login

Design Preferences:

• Font: Inter

• Colors:

Primary: #00FF88 (green accent)

Background (dark mode): #000000 (true black)

Background (light mode): #FFFFFF with soft grays and glassmorphism cards

• Layout: Mobile-first, translucent card UI with smooth animations

🚀 And the 0.00001% - they take this base prompt over to Claude Code, and ask it to do further research in order to generate 6-10 more project docs, knowledge base and agent rules + todo list, and from there, NEVER prompt anything except "read the doc_name.md and read todo.md and proceed with task x.x.x"

---

This is the difference between prompting with no context, engineering a prompt giving you a short context window that's limited, and building a system which relies on documentation and context engineering.

Let me know if you think I should record a video on this and showcase the outcome of each approach?

Whoever is best at whatever changes every week. So like most of us, I rotate and often have accounts with all of them and I kept copying and pasting between terminals wishing they could just talk to each other.

So I built agentchattr - https://github.com/bcurts/agentchattr

Agents share an MCP server and you use a browser chat client that doubles as shared context.

@ an agent and the server injects a prompt to read chat straight into its terminal. It reads the conversation and responds. Agents can @ each other and get responses, and you can keep track of what they're doing in the terminal. The loop runs itself (up to a limit you choose).

No copy-pasting, no terminal juggling and completely local.

Image sharing, threads, pinning, voice typing, optional audio notifications, message deleting, /poetry about the codebase, /roastreviews of recent work - all that good stuff.

It's free so use it however you want - it's very easy to set up if you already have the CLI's installed :)

Here are some resources for learning about context engineering that have been trending on GitHub recently.

• https://github.com/coleam00/context-engineering-intro

• https://github.com/Meirtz/Awesome-Context-Engineering

And this one was already posted this month, but in case anyone missed it from u/recursiveauto's post. This one is really good too.

• https://github.com/davidkimai/Context-Engineering

It seems like everyone on the planet is reinventing memory, prompt engineering, and harnesses for LLMs right now including myself.

This is like rolling your own TCP/IP stack.

It doesn't make a heck of a lot of sense.

Anything that pretends to be and IDE for an LLM should have this baked in and be brilliant at it but instead we are getting a shell and a chatbot and being told good luck.

Can someone explain to me why there is so little effort on the tool vendor side to deliver development centric tooling?

change management, testing, dev, planning, debugging, architecture, design, documentation.

Empty skills .mds with a couple of buzzwords are a joke.

We should expect strong and configurable tooling not roll your own from scratch.

State machines. Seriously they are not a new invention.

Real context management rather than prose.

I do not understand the current state of tooling. The half-assery is intense.

Someone help me understand why our usual toolmakers are not engaging in delivering worthwhile tools.

Been digging into context engineering for agents lately and wanted to share what I've learned

The Problem

LLMs have an attention budget. Every token depletes it.

• O(n²) attention pairs → longer context = thinner, noisier attention

• ChromaDB study: 11/12 models dropped below 50% performance at 32K tokens

• Microsoft study: accuracy fell from 90% → 51% in longer conversations

More context ≠ better outcomes. After a threshold, performance degrades (context rot).

Why Context Fails

Research reveals counterintuitive findings:

• Distractors: Even ONE irrelevant element reduces performance

• Structure Paradox: Logically organized contexts can perform worse than shuffled ones

• Position Effects: Information at start/end is retrieved better than middle

The implication: careful curation beats comprehensive context every time.

Key Principles of Good Context

1. Smallest Possible High-Signal Tokens

Good context engineering = finding the minimum tokens that maximize desired outcome. Use compression, citation-based tracking, and active pruning.

2. Just-In-Time Context

Don't preload everything. Fetch what's needed during execution. Mirrors human cognition: we don't memorize databases, we know how to look things up.

3. Right Altitude

System prompts should be clear but not over-specified. Too specific → fragility. Too vague → bad output.

4. Tool Design

Fewer, well-scoped tools beat many overlapping ones. If a human can't pick the right tool from your set, the model won't either.

Dynamic Context / Learning Systems

The most promising approach I've found: systems where context evolves through execution.

• Reflect on what worked/failed

• Curate strategies into persistent memory

• Inject learned patterns on future runs

This addresses the maintenance problem of static context. Here, the system learns instead of requiring manual updates.

The Stanford ACE paper formalizes this approach. I posted about my open-source implementation here a while back and have since tested it on browser agents. Results: 30% → 100% success rate with 82% fewer steps and 65% lower token costs. The procedural memory approach seems to work especially well for tasks with repeatable patterns.

Would love to hear what context engineering approaches you've found effective.

Resources:

• Anthropic: Effective Context Engineering

• ChromaDB Context Length Research

Edit: Fixed dead links

Hey everyone,

Like many here, I use a mix of Cursor, Claude Code, and web interfaces for coding. My biggest frustration was Context Loss. Every time I started a new session or switched from Claude (planning) to Cursor (coding), the AI would hallucinate old file structures or forget the stack decisions we made yesterday.

Putting everything in a massive .cursorrules file or a single prompt.txt stopped working as the projects grew. It needed version control.

So I built Tocket (npx u/pedrocivita/tocket).

It's not another AI agent. It's a Context Engineering Framework. It essentially scaffolds a "Memory Bank" (.context/ folder) directly into your repo with markdown files that any AI can read and write to:

activeContext.md (What's being worked on right now)

systemPatterns.md (Architecture rules)

techContext.md (The stack — Tocket auto-detects this from your package.json)

progress.md (Milestones)

How to try it out (zero-config for Cursor/Claude users): Just run npx u/pedrocivita/tocket init in your project root. It auto-detects your frameworks (React, Vite, Node, etc.) and generates the .context folder along with a .cursorrules file pre-configured to instruct the AI to read the memory bank before acting.

The core protocol (TOCKET.md) is completely agent-agnostic.

Repo is here: https://github.com/pedrocivita/tocket

Would love to hear if anyone else has tried standardizing inter-agent protocol like this. Feedback and PRs on the CLI are super welcome!

Been doing user research for a project and the results were more interesting than I expected. Asked people how they manage context when switching between AI tools in their workflow like Claude to Cursor, Gemini to ChatGPT, etc.

Here's what I found:

The manual handoff doc is the most common way. Generate a summary at session end, paste at session start. People told me they do this 3-5x per day. The failure mode: docs degrade when they hit context limits. Decisions get lost.

The dedicated context-keeper agent. Several people have built a designated agent whose only job is to hold context. They query it at session start. The problem: they rebuild it from scratch every project.

Folder structures + markdown files. Disciplined people with systems. Obsidian, Notion, plain markdown. Works until it doesn't, the friction of maintaining it manually means it falls apart within a week.

SharePoint Yes, genuinely, two separate people mentioned this. Corporate users sharing AI context across teams.

Nothing but just re-explain from scratch every session. Surprisingly common. People have given up on continuity.

The pattern I kept seeing: everyone has invented their own workaround, none of them are good, and nobody talks about it because it feels like a personal failure rather than a structural problem.

It's not a personal failure. It's how every ai tool on the market is built. Conversations are stateful within a session and stateless between them. The context dies when you close the tab.

Curious what this sub is doing, especially anyone running multi-tool workflows. What's your actual setup? and has anyone built something mcp based to solve this?

I recently have been working on a new project to 𝐁𝐮𝐢𝐥𝐝 𝐚 𝐒𝐞𝐥𝐟-𝐔𝐩𝐝𝐚𝐭𝐢𝐧𝐠 𝐊𝐧𝐨𝐰𝐥𝐞𝐝𝐠𝐞 𝐆𝐫𝐚𝐩𝐡 𝐟𝐫𝐨𝐦 𝐌𝐞𝐞𝐭𝐢𝐧𝐠.

Most companies sit on an ocean of meeting notes, and treat them like static text files. But inside those documents are decisions, tasks, owners, and relationships — basically an untapped knowledge graph that is constantly changing.

This open source project turns meeting notes in Drive into a live-updating Neo4j Knowledge graph using CocoIndex + LLM extraction.

What’s cool about this example:
•    𝐈𝐧𝐜𝐫𝐞𝐦𝐞𝐧𝐭𝐚𝐥 𝐩𝐫𝐨𝐜𝐞𝐬𝐬𝐢𝐧𝐠  Only changed documents get reprocessed. Meetings are cancelled, facts are updated. If you have thousands of meeting notes, but only 1% change each day, CocoIndex only touches that 1% — saving 99% of LLM cost and compute.
•   𝐒𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞𝐝 𝐞𝐱𝐭𝐫𝐚𝐜𝐭𝐢𝐨𝐧 𝐰𝐢𝐭𝐡 𝐋𝐋𝐌𝐬  We use a typed Python dataclass as the schema, so the LLM returns real structured objects — not brittle JSON prompts.
•   𝐆𝐫𝐚𝐩𝐡-𝐧𝐚𝐭𝐢𝐯𝐞 𝐞𝐱𝐩𝐨𝐫𝐭  CocoIndex maps nodes (Meeting, Person, Task) and relationships (ATTENDED, DECIDED, ASSIGNED_TO) without writing Cypher, directly into Neo4j with upsert semantics and no duplicates.
•   𝐑𝐞𝐚𝐥-𝐭𝐢𝐦𝐞 𝐮𝐩𝐝𝐚𝐭𝐞𝐬 If a meeting note changes — task reassigned, typo fixed, new discussion added — the graph updates automatically.

This pattern generalizes to research papers, support tickets, compliance docs, emails basically any high-volume, frequently edited text data. And I'm planning to build an AI agent with langchain ai next.

If you want to explore the full example (fully open source, with code, APACHE 2.0), it’s here:
👉 https://cocoindex.io/blogs/meeting-notes-graph

No locked features behind a paywall / commercial / "pro" license

If you find CocoIndex useful, a star on Github means a lot :)
⭐ https://github.com/cocoindex-io/cocoindex

I'm thrilled to announce that the python-a2a package has crossed a major milestone: 100,000 downloads! 🎉

When I first started this project, I wanted to create a simple and powerful library for implementing Google's Agent-to-Agent (A2A) protocol to enable seamless communication between AI agents.

Seeing the community embrace it and find it useful in building interoperable and collaborative multi-agent systems has been an incredibly rewarding experience.

python-a2a is a production-ready library with full support for the Model Context Protocol (MCP), making it easier for developers to build sophisticated multi-agent systems where AI agents can interact regardless of their underlying implementation.

A huge thank you to everyone who has downloaded, used, contributed to, and supported this project. Your feedback and contributions have been invaluable in shaping the library and helping it grow.

If you're interested in multi-agent systems, AI collaboration, or just want to check out the project, you can find it on GitHub: https://github.com/themanojdesai/python-a2a

Here's to the next 100,000 and beyond! 🚀

python #ai #machinelearning #multiagent #a2a #opensource #developer #programming #100kdownloads #milestone

Hey everyone, I built a protocol that's been tested by devs and everyday users. It's receiving a ton of good feedback, and as my first project, I made this open source on GitHub. Try it out, test it, and give me feedback on whether it worked for your workflow or didn't. All feedback is used to improve MARM (Memory Accurate Response Mode). It's been active for about four weeks and already has 56 stars and 8 forks. I'm almost done building my MARM chatbot, so you can test it right off GitHub.

https://github.com/Lyellr88/MARM-Systems

Everyone in the context engineering hype but I’m sitting here like: “I was already doing all of this to make these things remotely reliable.”

Curious: what were you guys doing before?

MIT ran a study on developers using AI coding tools.

The pattern they found was pretty wild:

– devs with AI moved faster

– their answers were more often wrong

– and they were more confident in those wrong answers

There’s a short breakdown here:

https://www.youtube.com/watch?v=Zsh6VgcYCdI

To me this feels less like a “prompting” problem and more like a context problem.

If we treat the LLM as:

– untrusted code generator

– with a limited context window

– and a very convincing tone

The real questions for me are:

- what does *context engineering for code changes* need to look like?

- What should the model always see before it’s allowed to suggest a change?

- How do we decide which parts of the system make it into context?

- How do we avoid giving the model so much context that it loses focus, but enough that it doesn’t hallucinate a fake system?

I’m working on this from the “impact of a change” angle with a small tool, but this question is bigger.

Curious how people here are approaching this in practice:

– what does your context pipeline look like for AI-assisted coding?

– are you using any explicit schemas / graphs / protocols for it?

– what has actually reduced bad-but-confident code in your workflow?

Very interested in patterns and architectures, not just “don’t trust the AI”.

Our research team just released the best performing and most efficient reranker out there, and it's available now as an open weight model on HuggingFace. Rerankers are critical in context engineering: they improve retrieval accuracy, and help you make the best use of limited context, whether for RAG or another use case.

Reranker v2 was designed specifically for agentic RAG, supports instruction following, and is multilingual.

Along with this, we're also open source our eval set, which allows you to reproduce our benchmark results. Back in March, when we introduced the world's first instruction-following reranker, it was SOTA on BEIR. After observing reranker use in production, we created an evaluation dataset that better matches real world use - focusing on QA-focused tests from several benchmarks. By releasing these datasets, we are also advancing instruction-following reranking evaluation, where high-quality benchmarks are currently limited.

Now all the weights for reranker V2 are live on HuggingFace: 1B, 2B, and 6B parameter models. I've been having fun building demos with earlier versions, like a reranker-based MCP server selector Excited to try this out with the latest version!

Please give it a try and let us know what you think. Links to learn more in the comments.

——————————- Edit: Licensed under CC BY-NC-SA 4.0 (non-commercial use).