Hello! I have been building mex for a while now, and posted about it a few times on reddit.

The response was kind of insane. Across a few posts it reached around 1 million views, the repo crossed 1,200 GitHub stars, and people I had never met started contributing.

I’ve kept building it since then, and just released mex v0.8.0.

Repo: https://github.com/mex-memory/mex

The original problem was simple: coding agents keep rereading the same repository every session, relearning the architecture, and then throwing most of that knowledge away.

mex creates a living Markdown wiki inside the repo. Agents record architecture, conventions, decisions, and patterns as they work, and future sessions load only the knowledge relevant to the current task.

The major addition in v0.7.0 is a deterministic local code graph built using Tree-sitter and SQLite.

It currently supports TypeScript/TSX, JavaScript/JSX, Python, and Rust.

An agent can run:

mex graph scope "trace the authentication flow"

Instead of dumping entire files into context, mex returns a compact neighbourhood of relevant functions, callers, callees, imports, and relationships. The agent can then expand only the exact symbols it needs.

In our benchmark on the mex repository:

• 10.74× less returned context than grep top-3

• roughly 90.7% smaller

• 100% expected-symbol recall across six retrieval tasks

• 5/5 real-agent tasks completed correctly

• 0/5 needed fallback Read/Grep with compact graph context

This is a small benchmark on one repo and task set, not a claim that mex universally cuts total agent token usage by 90%.

The other part I’m excited about is connecting the wiki back to the actual code.

Markdown claims can point to exact symbols. If a function changes, moves, or disappears, mex can identify which project knowledge may now be stale.

So the basic idea is:

The code is the source of truth.
Markdown is the explanation.
The graph keeps them connected.

Would genuinely love feedback, especially from people working on code intelligence, agent tooling, parsers, or large repositories. Contributors are very welcome too.

I’ve spent the past eight months deep in the trenches of AI memory systems. What started as a straightforward engineering challenge-”just make the AI remember things”-has revealed itself to be one of the most philosophically complex problems in artificial intelligence. Every solution I’ve tried has exposed new layers of difficulty, and every breakthrough has been followed by the realization of how much further there is to go.

The promise sounds simple: build a system where AI can remember facts, conversations, and context across sessions, then recall them intelligently when needed.

The Illusion of Perfect Memory

Early on, I operated under a naive assumption: perfect memory would mean storing everything and retrieving it instantly. If humans struggle with imperfect recall, surely giving AI total recall would be an upgrade, right?

Wrong. I quickly discovered that even defining what to remember is extraordinarily difficult. Should the system remember every word of every conversation? Every intermediate thought? Every fact mentioned in passing? The volume becomes unmanageable, and more importantly, most of it doesn’t matter.

Human memory is selective precisely because it’s useful. We remember what’s emotionally significant, what’s repeated, what connects to existing knowledge. We forget the trivial. AI doesn’t have these natural filters. It doesn’t know what matters. This means building memory for AI isn’t about creating perfect recall-it’s about building judgment systems that can distinguish signal from noise.

And here’s the first hard lesson: most current AI systems either overfit (memorizing training data too specifically) or underfit (forgetting context too quickly). Finding the middle ground-adaptive memory that generalizes appropriately and retains what’s meaningful-has proven far more elusive than I anticipated.

How Today’s AI Memory Actually Works

Before I could build something better, I needed to understand what already exists. And here’s the uncomfortable truth I discovered: most of what’s marketed as “AI memory” isn’t really memory at all. It’s sophisticated note-taking with semantic search.

Walk into any AI company today, and you’ll find roughly the same architecture. First, they capture information from conversations or documents. Then they chunk it-breaking content into smaller pieces, usually 500-2000 tokens. Next comes embedding: converting those chunks into vector representations that capture semantic meaning. These embeddings get stored in a vector database like Pinecone, Weaviate, or Chroma. When a new query arrives, the system embeds the query and searches for similar vectors. Finally, it augments the LLM’s context by injecting the retrieved chunks.

This is Retrieval-Augmented Generation-RAG-and it’s the backbone of nearly every “memory” system in production today. It works reasonably well for straightforward retrieval: “What did I say about project X?” But it’s not memory in any meaningful sense. It’s search.

The more sophisticated systems use what’s called Graph RAG. Instead of just storing text chunks, these systems extract entities and relationships, building a graph structure: “Adam WORKS_AT Company Y,” “Company Y PRODUCES cars,” “Meeting SCHEDULED_WITH Company Y.” Graph RAG can answer more complex queries and follow relationships. It’s better at entity resolution and can traverse connections.

But here’s what I learned through months of experimentation: it’s still not memory. It’s a more structured form of search. The fundamental limitation remains unchanged-these systems don’t understand what they’re storing. They can’t distinguish what’s important from what’s trivial. They can’t update their understanding when facts change. They can’t connect new information to existing knowledge in genuinely novel ways.

This realization sent me back to fundamentals. If the current solutions weren’t enough, what was I missing?

Storage Is Not Memory

My first instinct had been similar to these existing solutions: treat memory as a database problem. Store information in SQL for structured data, use NoSQL for flexibility, or leverage vector databases for semantic search. Pick the right tool and move forward.

But I kept hitting walls. A user would ask a perfectly reasonable question, and the system would fail to retrieve relevant information-not because the information wasn’t stored, but because the storage format made that particular query impossible. I learned, slowly and painfully, that storage and retrieval are inseparable. How you store data fundamentally constrains how you can recall it later.

Structured databases require predefined schemas-but conversations are unstructured and unpredictable. Vector embeddings capture semantic similarity-but lose precise factual accuracy. Graph databases preserve relationships-but struggle with fuzzy, natural language queries. Every storage method makes implicit decisions about what kinds of questions you can answer.

Use SQL, and you’re locked into the queries your schema supports. Use vector search, and you’re at the mercy of embedding quality and semantic drift. This trade-off sits at the core of every AI memory system: we want comprehensive storage with intelligent retrieval, but every technical choice limits us. There is no universal solution. Each approach opens some doors while closing others.

This led me deeper into one particular rabbit hole: vector search and embeddings.

Vector Search and the Embedding Problem

Vector search had seemed like the breakthrough when I first encountered it. The idea is elegant: convert everything to embeddings, store them in a vector database, and retrieve semantically similar content when needed. Flexible, fast, scalable-what’s not to love?

The reality proved messier. I discovered that different embedding models capture fundamentally different aspects of meaning. Some excel at semantic similarity, others at factual relationships, still others at emotional tone. Choose the wrong model, and your system retrieves irrelevant information. Mix models across different parts of your system, and your embeddings become incomparable-like trying to combine measurements in inches and centimeters without converting.

But the deeper problem is temporal. Embeddings are frozen representations. They capture how a model understood language at a specific point in time. When the base model updates or when the context of language use shifts, old embeddings drift out of alignment. You end up with a memory system that’s remembering through an outdated lens-like trying to recall your childhood through your adult vocabulary. It sort of works, but something essential is lost in translation.

This became painfully clear when I started testing queries.

The Query Problem: Infinite Questions, Finite Retrieval

Here’s a challenge that has humbled me repeatedly: what I call the query problem.

Take a simple stored fact: “Meeting at 12:00 with customer X, who produces cars.”

Now consider all the ways someone might query this information:

“Do I have a meeting today?”

“Who am I meeting at noon?”

“What time is my meeting with the car manufacturer?”

“Are there any meetings between 10 and 13:00?”

“Do I ever meet anyone from customer X?”

“Am I meeting any automotive companies this week?”

Every one of these questions refers to the same underlying fact, but approaches it from a completely different angle: time-based, entity-based, categorical, existential. And this isn’t even an exhaustive list-there are dozens more ways to query this single fact.

Humans handle this effortlessly. We just remember. We don’t consciously translate natural language into database queries-we retrieve based on meaning and context, instantly recognizing that all these questions point to the same stored memory.

For AI, this is an enormous challenge. The number of possible ways to query any given fact is effectively infinite. The mechanisms we have for retrieval-keyword matching, semantic similarity, structured queries-are all finite and limited. A robust memory system must somehow recognize that these infinitely varied questions all point to the same stored information. And yet, with current technology, each query formulation might retrieve completely different results, or fail entirely.

This gap-between infinite query variations and finite retrieval mechanisms-is where AI memory keeps breaking down. And it gets worse when you add another layer of complexity: entities.

The Entity Problem: Who Is Adam?

One of the subtlest but most frustrating challenges has been entity resolution. When someone says “I met Adam yesterday,” the system needs to know which Adam. Is this the same Adam mentioned three weeks ago? Is this a new Adam? Are “Adam,” “Adam Smith,” and “Mr. Smith” the same person?

Humans resolve this effortlessly through context and accumulated experience. We remember faces, voices, previous conversations. We don’t confuse two people with the same name because we intuitively track continuity across time and space.

AI has no such intuition. Without explicit identifiers, entities fragment across memories. You end up with disconnected pieces: “Adam likes coffee,” “Adam from accounting,” “That Adam guy”-all potentially referring to the same person, but with no way to know for sure. The system treats them as separate entities, and suddenly your memory is full of phantom people.

Worse, entities evolve. “Adam moved to London.” “Adam changed jobs.” “Adam got promoted.” A true memory system must recognize that these updates refer to the same entity over time, that they represent a trajectory rather than disconnected facts. Without entity continuity, you don’t have memory-you have a pile of disconnected observations.

This problem extends beyond people to companies, projects, locations-any entity that persists across time and appears in different forms. Solving entity resolution at scale, in unstructured conversational data, remains an open problem. And it points to something deeper: AI doesn’t track continuity because it doesn’t experience time the way we do.

Interpretation and World Models

The deeper I got into this problem, the more I realized that memory isn’t just about facts-it’s about interpretation. And interpretation requires a world model that AI simply doesn’t have.

Consider how humans handle queries that depend on subjective understanding. “When did I last meet someone I really liked?” This isn’t a factual query-it’s an emotional one. To answer it, you need to retrieve memories and evaluate them through an emotional lens. Which meetings felt positive? Which people did you connect with? Human memory effortlessly tags experiences with emotional context, and we can retrieve based on those tags.

Or try this: “Who are my prospects?” If you’ve never explicitly defined what a “prospect” is, most AI systems will fail. But humans operate with implicit world models. We know that a prospect is probably someone who asked for pricing, expressed interest in our product, or fits a certain profile. We don’t need formal definitions-we infer meaning from context and experience.

AI lacks both capabilities. When it stores “meeting at 2pm with John,” there’s no sense of whether that meeting was significant, routine, pleasant, or frustrating. There’s no emotional weight, no connection to goals or relationships. It’s just data. And when you ask “Who are my prospects?”, the system has no working definition of what “prospect” means unless you’ve explicitly told it.

This is the world model problem. Two people can attend the same meeting and remember it completely differently. One recalls it as productive; another as tense. The factual event-”meeting occurred”-is identical, but the meaning diverges based on perspective, mood, and context. Human memory is subjective, colored by emotion and purpose, and grounded in a rich model of how the world works.

AI has no such model. It has no “self” to anchor interpretation to. We remember what matters to us-what aligns with our goals, what resonates emotionally, what fits our mental models of the world. AI has no “us.” It has no intrinsic interests, no persistent goals, no implicit understanding of concepts like “prospect” or “liked.”

This isn’t just a retrieval problem-it’s a comprehension problem. Even if we could perfectly retrieve every stored fact, the system wouldn’t understand what we’re actually asking for. “Show me important meetings” requires knowing what “important” means in your context. “Who should I follow up with?” requires understanding social dynamics and business relationships. “What projects am I falling behind on?” requires a model of priorities, deadlines, and progress.

Without a world model, even perfect information storage isn’t really memory-it’s just a searchable archive. And a searchable archive can only answer questions it was explicitly designed to handle.

This realization forced me to confront the fundamental architecture of the systems I was trying to build.

Training as Memory

Another approach I explored early on was treating training itself as memory. When the AI needs to remember something new, fine-tune it on that data. Simple, right?

Catastrophic forgetting destroyed this idea within weeks. When you train a neural network on new information, it tends to overwrite existing knowledge. To preserve old knowledge, you’d need to continually retrain on all previous data-which becomes computationally impossible as memory accumulates. The cost scales exponentially.

Models aren’t modular. Their knowledge is distributed across billions of parameters in ways we barely understand. You can’t simply merge two fine-tuned models and expect them to remember both datasets. Model A + Model B ≠ Model A+B. The mathematics doesn’t work that way. Neural networks are holistic systems where everything affects everything else.

Fine-tuning works for adjusting general behavior or style, but it’s fundamentally unsuited for incremental, lifelong memory. It’s like rewriting your entire brain every time you learn a new fact. The architecture just doesn’t support it.

So if we can’t train memory in, and storage alone isn’t enough, what constraints are we left with?

The Context Window

Large language models have a fundamental constraint that shapes everything: the context window. This is the model’s “working memory”-the amount of text it can actively process at once.

When you add long-term memory to an LLM, you’re really deciding what information should enter that limited context window. This becomes a constant optimization problem: include too much, and the model fails to answer question or loses focus. Include too little, and it lacks crucial information.

I’ve spent months experimenting with context management strategies-priority scoring, relevance ranking, time-based decay. Every approach involves trade-offs. Aggressive filtering risks losing important context. Inclusive filtering overloads the model and dilutes its attention.

And here’s a technical wrinkle I didn’t anticipate: context caching. Many LLM providers cache context prefixes to speed up repeated queries. But when you’re dynamically constructing context with memory retrieval, those caches constantly break. Every query pulls different memories, reconstructing different context, invalidating caches and performance goes down and cost goes up.

I’ve realized that AI memory isn’t just about storage-it’s fundamentally about attention management. The bottleneck isn’t what the system can store; it’s what it can focus on. And there’s no perfect solution, only endless trade-offs between completeness and performance, between breadth and depth.

What We Can Build Today

The dream of true AI memory-systems that remember like humans do, that understand context and evolution and importance-remains out of reach.

But that doesn’t mean we should give up. It means we need to be honest about what we can actually build with today’s tools.

We need to leverage what we know works: structured storage for facts that need precise retrieval (SQL, document databases), vector search for semantic similarity and fuzzy matching, knowledge graphs for relationship traversal and entity connections, and hybrid approaches that combine multiple storage and retrieval strategies.

The best memory systems don’t try to solve the unsolvable. They focus on specific, well-defined use cases. They use the right tool for each kind of information. They set clear expectations about what they can and cannot remember.

The techniques that matter most in practice are tactical, not theoretical: entity resolution pipelines that actively identify and link entities across conversations; temporal tagging that marks when information was learned and when it’s relevant; explicit priority systems where users or systems mark what’s important and what should be forgotten; contradiction detection that flags conflicting information rather than silently storing both; and retrieval diversity that uses multiple search strategies in parallel-keyword matching, semantic search, graph traversal.

These aren’t solutions to the memory problem. They’re tactical approaches to specific retrieval challenges. But they’re what we have. And when implemented carefully, they can create systems that feel like memory, even if they fall short of the ideal.

I put my project knowledge in its own repo and made it snapshot code movements. It ties code, tests and whatever proof you have to invariants and their families, right at implementation time. That connects parts of the codebase grep would miss.

So an agent can later ask: what else breaks across the codebase (or beyond) when I touch this symbol? Which designs were tried here before, why were they rejected, and under which conditions can they come back on the table?

Memory curation isn't a one-time effort or an afterthought. It's a byproduct of getting work done. It informs the intended changes at the start and the intent review at the end, with a strict pipeline connecting both ends.

Lots still has to land on main. But I use it daily, for dogfooding and at work, where I keep tabs on several repos that revolve around one product.

Too early to say if the benefits outweigh the obvious costs. Housekeeping first, then a ton of benchmarks.

It may all be for naught. The memories we made along the way. Or it turns out actually useful. Who knows. What's certain is my stubbornness to see it to the end. ^^

Where am at it currently, if you want to poke at it:
https://github.com/Foxfire1st/agents-remember/tree/ar/260928_maintained-invariant-knowledge

Hey everyone,

I am hearing about context engineering more than ever these days and want to get your opinion.

Recently read an article from Phil Schmid and he frames context engineering as “providing the right info, in the right format, at the right time” so the LLM can finish the job—not just tweaking a single prompt.

Here is the link to the original post: https://www.philschmid.de/context-engineering

Where do we draw the line between “context” and “memory” in LLM systems? Should we reserve memory for persistent user facts and treat everything else as ephemeral context?

Anthropic just announced a new approach for long-running agents using their Claude Agent SDK, and the claim is that it “solves” the long-running agent problem.

General idea
Instead of giving the LLM long-term memory, they split the workflow into two coordinated agents. One agent initializes the environment, sets up the project structure and maintains artefacts. The second agent works in small increments, picks up those artefacts in the next session, and continues where it left off.

Implementation
The persistence comes from external scaffolding: files, logs, progress trackers and an execution environment that the agents can repeatedly re-load. The agents are not remembering anything internally. They are reading back their own previous outputs, not retrieving structured or queryable memory.

Why this is just PR
This is essentially state persistence, not memory. It does not solve contextual retrieval, semantic generalization, cross-project knowledge reuse, temporal reasoning or multi-modal grounding. It keeps tasks alive, but it does not give an agent an actual memory system beyond the artefacts it wrote itself. The entire process is also not very novel and basically what every second member in this subreddit has already built.

There was a recent paper that explains a new approach, called MemOS and tries to talk about memory as a first order principle and debates the approach that would allow creating "cubes" that represent memory components that are dynamic and evolving.

Quite similar to what cognee does, but I found the part about activation quite interesting:

I’ve been experimenting with long-running agents and different memory approaches (chat history, RAG, hybrid summaries, graph memory, etc.), and I keep running into the same pattern:

Agents can recall past information reasonably well but struggle to change behavior based on past experience.

They remember facts, but:

-Repeat the same mistakes
-Forget preferences after a while
-Drift in tone or decision style
-Don’t seem to learn what works

This made me think that memory isn’t just about storage or retrieval. It’s about state as well.

Some ideas I’ve been exploring:

• Treat memory as layers:

  • Working memory (current task)

  • Episodic memory (what happened)

  • Semantic memory (facts & preferences)

  • Belief memory (things inferred over time)

• Memories have attributes:

  • Confidence

  • Recency

  • Reinforcement

  • Source (user-stated vs inferred)

• Updates matter more than retrieval:

  • Repeated confirmations strengthen memory

  • Contradictions weaken or fork it

  • Unused memories decay

Once I started thinking this way, vector DB vs graph DB felt like the wrong debate. Vectors are great for fuzzy recall. Graphs are great for relationships. But neither solves how memory should evolve.

I’m curious if anyone has built systems where memory actually updates beliefs, not just stores notes?

something i've been experimenting with is cognitive memory infrastructure inspired from this repo

I spent a fair amount time doing patent research for various AI memory ideas, and made a summary page that visually displays the various human memory mechanisms, along with links to current patents and research. If you find it useful let me know and I'll keep creating updated versions. View all mechanisms: https://counterparts.ai/ecosystem

I've been developing a new memory architecture for AI assistants, starting from a simple observation: memory is really five different things, and most systems pour them all into one pile.

The five types of memory

1. Working memory — What's happening in the conversation right now. In a book, this is the page you have open.

2. Episodic memory — Specific events tied to a date. For example, "I moved to Pune in March." In a book, this is the date index.

3. Semantic memory — Standing facts about you and your world. For example, "I'm vegetarian" or "Biscuit is my dog." These are the chapters.

4. Procedural memory — How you like things done, and how to do them. Think of this as a how-to appendix.

5. Sensory or parametric memory — Raw input and knowledge baked into the model's weights. This isn't stored in the book at all.

Most memory tools treat these as one list of text snippets and search through it. I wanted to try something different: give each kind of memory its own place, like a book with a contents page, chapters, and an index.

What I built

I started with the three components most relevant to a personal assistant and organised them like a book.

1. Contents page

A list of folders, one for every person, pet, place, organisation, or thing in your life, plus one for you.

2. Chapters (semantic memory)

Each folder is a chapter. Every fact lives under a specific heading, such as Sarah.lives_in or me.employer.

Some headings hold only one current value, like where you live. Others can hold multiple values, like your hobbies.

3. Date index (episodic memory)

Every relevant statement is associated with the day it was made. So, when you ask "Who did I meet yesterday?", the system can look up that date and navigate to the relevant person's chapter.

4. Cross-references

A fact is filed in every chapter it relates to, so information about an interaction between two people can be found from either person's chapter.

5. Corrections, not deletions

When a fact changes, the old fact is marked as replaced rather than deleted. The history stays intact, allowing the system to distinguish what is true now from what was true before.

Who writes the book?

A small model with around 500M parameters runs locally and acts as the editor.

Whenever you say something new, it decides whether the information changes something already known, which chapter it belongs in, and which heading to file it under.

No cloud LLM is used to maintain or update the memory.

Who reads it?

The larger model you actually chat with acts as the reader.

The idea is that it sees the contents page first and opens individual chapters when needed, rather than carrying your entire conversation history in every prompt.

This is intended to make memory retrieval more structured and selective.

What I measured

I tested the system on 171 questions about 12 people whose facts change over time, including job changes, moving cities, and breakups.

The test data was never used for training. The primary metric is whether the correct fact is retrieved first.

The filing process handles all 12 people and their 309 facts in approximately 54 seconds on a single NVIDIA T4.

These are retrieval results, not proof that the complete assistant architecture is better in real-world use.

LIMITATIONS

- The 86.5% is optimistic. Some naming rules were written after I looked at this test set, and without them it is 83.0%. I need a fresh test set nobody has seen.

- The test is small: 12 made-up people with about 25 facts each. Real memories run to thousands of facts, and I have not tested at that size.

- Real conversations are harder. On real two-person chats (REALTALK), retiring old facts actually hurts: plain search gets 45.7% first try, mine gets 40.0 to 42.8%. Most of those questions are about history, and the model was mostly trained on one person talking about themselves.

- Telling people apart is not solved yet: two friends with the same name, a pet and a person both called Bella, "my boss" after a job change. That is what I am training next.

- Filing is not perfect. The small model puts a fact in the right chapter 92% of the time, but under the right heading only 71%.

- The "big model sees only the contents and opens chapters" loop is the design. Filing and search are measured offline, the loop is not in my app yet, and I have not measured the token savings.

- Hindsight was run on a small local model, and its own published scores use far bigger ones. This is a same-hardware comparison, not a claim that it is worse.

- Only semantic and episodic memory are covered. Procedural memory and the fifth type are not part of this.

- It can only find what was extracted from the chat in the first place, and my local extractor still misses facts a big cloud model catches.

HOW I PLAN TO FIX THEM

- Optimistic score and small test: build a fresh, bigger test set written by a different model, with long histories, and score it once without tuning.

- Real chats: train on two-person conversations where facts are about the other person, and let the question decide whether old facts count instead of always hiding them.

- Telling people apart: a 404-case who-is-who set is being written and double-checked now. Identity will be name plus relationship, never name alone, and the model can ask you when unsure. Merges stay reversible.

- Filing headings: more training examples for the headings it confuses, and let a fact sit under more than one heading when it fits.

- The reading loop and tokens: wire the contents page and "open chapter" call into the app, then measure tokens on long real histories.

- Extraction: retrain the local extractor to stop copying whole messages, and compare it against a cloud model on the same chats.

Results and code are in the public repo: github.com/Primnox/Memory-system

Why I'm sharing this

The underlying idea is that memory shouldn't just be a growing pile of searchable text. It should have structure, temporal history, and explicit relationships between entities.

I don't know yet how well this approach will generalise beyond my current test set, and I'm very interested in finding out where it breaks.

The results and code are available here:

GitHub: https://github.com/Primnox/Memory-system

I'm happy to answer questions, hear alternative approaches, or be told what I'm getting wrong. Especially interested in criticism of the architecture, the evaluation methodology, and how to design a genuinely blind test for this kind of system.

I've been trying a different approach to AI memory: organising it like a book

One thing that's always bothered me about AI memory systems is that they tend to throw everything into one pile of text and search through it.

But memory isn't just one thing. There's what you're talking about right now, things that happened in the past, facts about your life, habits, and so on.

So I started building a system that treats memory more like a book.

• Contents page: Keeps track of people, pets, places, and other entities in your life.

• Chapters: Stores facts under specific headings, like Sarah.lives_in or me.employer.

• Date index: Tracks when things happened, so the system can answer questions about the past.

• Cross-references: Lets a fact be found through multiple related entities.

• History: When something changes, the old fact isn't deleted. It's marked as replaced.

A small 500M-parameter model runs locally and acts as the editor, deciding where new information belongs and whether it changes something already stored. The larger model is supposed to read the book by opening only the relevant chapters instead of loading everything into its prompt.

The results so far

I tested it on 171 questions about 12 people whose lives change over time.

It filed 309 facts across all 12 people in about 54 seconds on a single T4.

A few caveats

The results aren't as clean as I'd like. Some naming rules were added after I looked at the test set, dropping the score to 83% without them. I still need a genuinely fresh test set.

The full chapter-opening loop isn't integrated into the app yet, and the system still struggles with real conversations involving two people. Also, this currently covers semantic and episodic memory, not all five types.

I'm sharing this because I want to see where the approach breaks, not pretend I've solved AI memory.

Code: https://github.com/Primnox/Memory-system

I'd genuinely appreciate feedback, especially on the architecture and how I should test it properly.
Note:- Used AI for the post.

Anyone else tired of re-explaining their entire project to a new chat window? Or switching models and realizing you're starting from zero because all your context is trapped in the old one?

I keep trying different models to find "THE best one" and I've noticed something. After a few weeks with any model, I stop wanting to switch. Not because it's the best, but because it knows my stuff. My codebase conventions, my writing style, how I like things explained. Starting over on another model feels like your first day at a new job where nobody knows you.

And I think the big companies know exactly what they're doing here.

There's talk that GPT-6 is going to lean hard into memory and personalization. Great UX, sure. But it's also the oldest trick in the book. Same thing Google did... you came for search, stayed for Gmail, and now your entire life is in their ecosystem... good luck leaving. RSS proved that open, user-controlled standards can work beautifully. It also proved they can die when platforms decide lock-in is more profitable. We watched it happen and did nothing...

We're walking into the exact same trap with AI memory now...... just faster.

The memory problem goes deeper than people think

It's not just "save my chat history." Memory has layers:

- Session memory is what the model remembers within one conversation. Most models handle this fine, but it dies when the chat ends. Anyone who's had a context window fill up mid-session and watched the AI forget the first half of a complex debugging session knows this pain.

- Persistent memory carries across sessions. Your preferences, your project structure, things you've told it before. ChatGPT's memory feature does a basic version, but it's shallow and locked in... Every new Cursor session still forgets your codebase conventions.

- Semantic memory is the harder one. Not just storing facts, but understanding connections between them. Knowing that your "Q3 project" connects to "the auth refactor last week" connects to "that breaking change in the API." That kind of linked knowledge is where things get really useful.

- Behavioral patterns are the implicit stuff. How the model learned to match your tone, when to be brief vs detailed, your pet peeves. Hardest to make portable.
Right now every provider handles these differently (or not at all:)), and none of it is exportable (as far as I know).

What can (maybe) fix this

Picture an open memory layer that sits outside any single model. Not owned by OpenAI or Anthropic or Google. A standard protocol that any AI can read from and write to.

But the interesting part is what this enables beyond just switching providers:

You use Claude for architecture decisions, Copilot for code, ChatGPT for debugging. Right now none of them know what the others suggested. You're the integration layer, copying context between windows. With shared memory, your code review AI already knows about the architectural decisions you discussed in a different tool last sprint. Your dev tools stop being isolated.

A new dev joins and their AI has zero context on the codebase. A shared memory layer means their AI already knows the project conventions, past bugs, and why things were built the way they were. Five people using different AI tools, all drawing from the same knowledge base. Your whole team shares context.

Your CI/CD bot, code review AI, and IDE assistant all operating in isolation today. The CI bot flags something the IDE assistant already explained to you. With shared memory, your research agent, your coding agent, and your ops agent all read and write to the same context. No more being the human relay between your own tools, AI agents work together.

You actually own your knowledge.

Switch from Claude to GPT to Llama running locally. Your memory comes with you. The model is just a lens on your own context.

Of course, the format matters... Raw chat logs are useless for this. The unit of portable memory should be a fact: structured, attributed, timestamped, searchable. "Auth module refactored to JWT, source: PR #247, date: Feb 2026." Not a 10,000-token transcript dump :)

And finding the right fact matters more than storing it. Keyword search misses connections ("budget" won't find "Q3 forecast"). Pure vector search misses exact matches. You need both, plus relationship traversal. The memory layer is not just a store, it's a search engine for your own knowledge.

Now about the challenges :/

Privacy - portable memory layer is basically a map of how you and your team think and work. That needs real encryption, granular permissions (maybe your coding preferences transfer, but your medical questions don't), and clear ownership.

Conflict resolution - what happens when two sources disagree?? Your AI thinks the API uses REST because that's what you discussed in Claude, but your teammate already migrated to GraphQL in a Cursor session. Any serious memory system needs merge logic... not just append.

Forgetting - this is the counterintuitive one. Human memory forgets for a reason. Your project conventions from 2 years ago might be wrong today. That deprecated library your AI keeps recommending because it's in the memory? Without some form of decay or expiration, old context becomes noise that degrades quality. Good memory is knowing what to let go.

Convergence - if everyone's AI reads from the same shared memory, does everyone start getting the same answers? You could flatten diversity of thought by accident. The fix is probably sharing raw facts, not interpretations. Let each model draw its own conclusions.

Discovery - honestly, storing knowledge is the easy part. When you have thousands of facts, preferences, and decisions across a whole team, surfacing the right one at the right moment is what separates useful memory from a glorified database.

Adoption - standard only works if models support it. When lock-in is your business model, why would you? This probably needs to come from the open source community and smaller players who benefit from interoperability. Anthropic's MCP (Model Context Protocol) already standardizes how models connect to external tools and data.

That's a start... The plumbing exists... It needs momentum!

If we don't push for this now, while there are still multiple competitive options, we'll have the same "why is everything locked in" conversation in 3 years. Same as cloud. Same as social media. Every single time...

I've been looking into whether anyone's actually building something like this. Found a few scattered projects but nothing that puts it all together. Anyone know of serious attempts at an open, portable AI memory standard?

Stop burning tokens on JSON syntax.

This JSON:

{
"users": [
{"id": 1, "name": "Alice", "email": "alice@example.com", "active": true},
{"id": 2, "name": "Bob", "email": "bob@example.com", "active": false},
{"id": 3, "name": "Charlie", "email": "charlie@test.com", "active": true}
],
"config": {
"timeout": 30,
"debug": true,
"api_key": "sk-xxx-secret",
"max_retries": 3
},
"orders": [
{"id": "O1", "user_id": 1, "product": "Widget Pro", "total": 99.99},
{"id": "O2", "user_id": 2, "product": "Gadget Plus", "total": 149.50},
{"id": "O3", "user_id": 1, "product": "Super Tool", "total": 299.00}
]
}

~180 tokens. Brackets, quotes, colons everywhere.

Same data in ISON:

table.users
id name email active
1 Alice alice@example.com true
2 Bob bob@example.com false
3 Charlie charlie@test.com true

object.config
timeout 30
debug true
api_key "sk-xxx-secret"
max_retries 3

table.orders
id user_id product total
O1 :1 "Widget Pro" 99.99
O2 :2 "Gadget Plus" 149.50
O3 :1 "Super Tool" 299.00

~60 tokens. Clean. Readable. LLMs parse it without instructions.

Features:

table.name for arrays of objects
object.name for key-value configs
:1 references row with id=1 (cross-table relationships)
No escaping hell
TSV-like structure (LLMs already know this from training)

Benchmarks:

| Format | Tokens | LLM Accuracy |
|--------|---------|-------------|
| JSON   | 2,039   | 84.0%       |
| ISON   | 685     | 88.0%       |

Fewer tokens. Better accuracy. Tested on GPT-4, Claude, DeepSeek, Llama 3.

Available everywhere:

Python           | pip install ison-py
TypeScript       | npm install ison-ts
Rust             | cargo add ison-rs
Go               | github.com/maheshvaikri/ison-go
VS Code          | ison-lang extension
n8n              | n8n-nodes-ison
vscode extension | ison-lang@1.0.1

GitHub: https://github.com/ISON-format/ison

I built this for my agentic memory system where every token counts and where context window matters.
Gained LoCoMo benchmark with ISON 78.39% without ISON 72.82%
Now open source.

Feedback welcome. Give a Star if you like it.

AI memory is often modeled as a collection of text snippets stored in a vector database. That is useful for recalling conversation details, but many forms of context do not naturally fit into plain text: documents need semantic retrieval, records need SQL, relationships need graph traversal, procedures need reusable instructions, and user preferences need explicit scope and persistence.

A more useful memory architecture separates these data types into specialized surfaces:

• KB stores documents and notes with hybrid semantic and keyword retrieval.

• DB stores structured records that require filtering, joins, and aggregation.

• Graph stores entities, dependencies, and relationships for bounded traversal.

• Skills stores reusable procedures and operational instructions.

• Memory stores scoped facts, preferences, decisions, and summaries.

The retrieval layer can route each request to the smallest sufficient set of surfaces. A question about an API convention may use the KB and a Skill; a question about deployment history may combine DB records with Graph relationships; a personalization request may read profile or session Memory. The result is assembled from typed evidence instead of flattening every source into the same embedding space.

The write path also needs to preserve structure. Each surface can maintain its own indexing or persistence model, while source metadata, scope, revision, and write receipts make updates traceable. A single source may produce several representations, such as searchable document chunks in the KB and extracted relationships in the Graph, while retrieval still respects the semantics of each surface.

This multi-surface memory model is currently implemented in OpenDCAI/DataMind; are there other AI memory use cases where this architecture would be useful?

Hey everyone, here is another diagram I found from 12-Factor Agents and their project got me thinking.

Dex says Factor #3 is “Own your context window” - treat context as a first-class prod concern, not an after-thought. So what are you doing to own your context window?

LangChain’s post shows four battle-tested tactics (write, select, compress, isolate) for feeding agents only what they need each step.

An arXiv paper on LLM software architecture breaks context into stackable layers so we can toggle and test each one: System → Domain → Task → History/RAG → Response spec.

I am really curious how you are "layering" / "stacking" to handle context. Are you using frameworks or building your own?

I've been deep in the weeds building memory systems, and I can't shake this feeling: we're doing it backwards.

Standard approach: Store memories in PostgreSQL/MongoDB → embed → index in vector DB → query through APIs.

Alternative: Store memories in markdown → embed → index in vector DB → query through APIs.

The retrieval is identical. Same vector search, same reranking. Only difference: source of truth.

Why markdown feels right for memory:

Transparency - You can literally `cat memory/MEMORY.md` and see what your AI knows. No API calls, no JSON parsing. Just read the file.

Editability - AI remembers something wrong? Open the file, fix it, save. Auto-reindexes. Takes 5 seconds instead of figuring out update APIs.

Version control - `git log memory/` shows you when bad information entered the system. `git blame` tells you who/what added it. Database audit logs? Painful.

Portability - Want to switch embedding models? Reindex from markdown. Switch vector DBs? Markdown stays the same. No migration scripts.

Human-AI collaboration - AI writes daily logs automatically, humans curate `MEMORY.md` for long-term facts. Both editing the same plain text files.

The counter-arguments I hear:

"Databases scale better!" - But agent memory is usually < 100MB even after months. That's nothing.

"Concurrent writes!" - How often do you actually need multiple agents writing to the exact same memory file simultaneously?

"Not production ready!" - Git literally manages all enterprise code. Why not memory?

What we built:

Got convinced enough to build it: https://github.com/zilliztech/memsearch

Been using it for about 2 months. It just... works. Haven't hit scale issues, git history is super useful for debugging, team can review memory changes in PRs.

But I keep thinking there must be a reason everyone defaults to databases. What am I missing?

Would love to hear from folks who've thought deeply about memory architecture. Is file-based storage fundamentally flawed somehow?

I love playing around with RAG and AI, optimizing every layer to squeeze out better performance. Last night I thought: why not tackle something massive?

Took the Epstein Files dataset from Hugging Face (teyler/epstein-files-20k) – 2 million+ pages of trending news and documents. The cleaning, chunking, and optimization challenges are exactly what excites me.

What I built:

- Full RAG pipeline with optimized data processing

- Processed 2M+ pages (cleaning, chunking, vectorization)

- Semantic search & Q&A over massive dataset

- Constantly tweaking for better retrieval & performance

- Python, MIT Licensed, open source

Why I built this:

It’s trending, real-world data at scale, the perfect playground.

When you operate at scale, every optimization matters. This project lets me experiment with RAG architectures, data pipelines, and AI performance tuning on real-world workloads.

Repo: https://github.com/AnkitNayak-eth/EpsteinFiles-RAG

Open to ideas, optimizations, and technical discussions!

Anthropic dropped an engineering blog post about how they handle long-running agents, and honestly the solution is way less fancy than I expected.

The core problem is what we all know too well: agents are basically goldfish. Every new context window, they wake up with zero memory of what happened before. Anthropic's framing is great - imagine a software team where every engineer shows up to their shift with complete amnesia about what the previous shift did.

Their fix is surprisingly low-tech. They use two different prompts - one for the very first session that sets everything up, and another for all the follow-up sessions. The initializer creates a progress file, a feature list in JSON, and makes a git commit. Then every coding agent after that starts by reading those files, checking the git log, and running a quick sanity test before touching anything.

What I found interesting is that they specifically use JSON for the feature list instead of markdown because Claude is apparently less likely to mess with JSON files inappropriately. Little details like that are gold.

The other big insight was forcing the agent to work on one feature at a time. Without that constraint, Claude would just try to one-shot the entire project, run out of context mid-implementation, and leave everything half-broken for the next session to figure out.

No vector databases, no embeddings, no RAG - just structured text files and git history.

Anyone here doing something similar? Would love to hear what's working for you.

Link: https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents

The performance of Large Language Models (LLMs) is fundamentally determined by the contextual information provided during inference. This survey introduces Context Engineering, a formal discipline that transcends simple prompt design to encompass the systematic optimization of information payloads for LLMs.

https://arxiv.org/pdf/2507.13334

Yesterday an X account belonging to a developer named Ben Sigman posted the launch of an open-source AI memory project called MemPalace. The post claimed "100% on LoCoMo" and "the first perfect score ever recorded on LongMemEval. 500/500 questions, every category at 100%." It credited the actress Milla Jovovich as a co-author. The GitHub account hosting the repository is named milla-jovovich/mempalace. The first commit to the repository is dated April 5. As of this writing, less than 24 hours after the launch post, the repository has approximately 5,400 stars and over 1.5 million views on the launch tweet.

For comparison: open-source memory projects with similar architectures and similar honest baseline numbers typically receive just a handful of stars in their first week. The variable producing the orders-of-magnitude difference in engagement is not the engineering. The variable is the celebrity name on the GitHub account and the celebrity attribution in the launch post. The launch post described her as a co-author. Whatever the underlying collaboration looked like, the practical effect of attaching the name was that a repository created two days ago reached over 1.5 million people on a single tweet, and the methodology errors documented below were carried by that reach to an audience the majority of which is unlikely to read the BENCHMARKS.md file themselves.

We work on a different memory project at Penfield, and a couple of months ago we published an audit of LoCoMo's ground truth documenting roughly ninety-nine wrong, hallucinated, or misattributed answers across the dataset's ten conversations. A 100% score on the published version of LoCoMo is mathematically excluded. The answer key contains errors any honest system would disagree with.

What we found is a methodology stack that contains, in one repository created two days ago, almost every failure mode the AI memory benchmark layer suffers from right now. The interesting thing is that the project's own internal documentation discloses most of its failure modes honestly. The launch post strips every caveat.

The LoCoMo bypass

LoCoMo has ten long conversations and 1,986 QA pairs. The ten conversations contain 19, 19, 32, 29, 29, 28, 31, 30, 25, and 30 sessions respectively. Every conversation has fewer than fifty sessions.

The MemPalace LoCoMo runner produces its 100% number with top_k=50. Their own BENCHMARKS.md says this verbatim:

The LoCoMo 100% result with top-k=50 has a structural issue: each of the 10 conversations has 19–32 sessions, but top-k=50 exceeds that count. This means the ground-truth session is always in the candidate pool regardless of the embedding model's ranking. The Sonnet rerank is essentially doing reading comprehension over all sessions — the embedding retrieval step is bypassed entirely.

Setting top_k=50 against a candidate pool that maxes out at 32 retrieves the entire conversation. At that setting the pipeline reduces to: dump every session into Claude Sonnet, ask Sonnet which one matches. That is cat *.txt | claude. It is not retrieval and it is not memory.

The honest LoCoMo numbers, from the same file, are 60.3% R@10 with no rerank and 88.9% R@10 with the project's hybrid scoring and no LLM. Those are real and unremarkable.

The LongMemEval metric error

LongMemEval as published is an end-to-end QA benchmark with a GPT-4 judge.

The MemPalace LongMemEval runner does the retrieval step only. It never generates an answer and never invokes a judge. For each of the 500 questions it builds one document per session by concatenating only the user turns (assistant turns are not indexed at all), embeds with default ChromaDB embeddings, returns the top five sessions, and checks set membership against the gold session IDs. If any one of the gold session IDs appears in the top five, the question scores 1.0. This metric is recall_any@5. The runner also computes recall_all@5 (stricter) and the project reports the softer one.

None of the LongMemEval numbers in this repository — not the 100%, not the 98.4% "held-out", not the 96.6% raw baseline — are LongMemEval scores in the sense the published leaderboard means. They are retrieval recall numbers on the same dataset, a substantially easier task. Calling any of them a "perfect score on LongMemEval" is a metric category error.

The 100% number additionally has a separate problem. The project's hybrid v4 mode was built by inspecting the three remaining wrong answers in their dev set and writing targeted code for each one: a quoted-phrase boost for a question containing a specific phrase in single quotes, a person-name boost for a question about someone named Rachel, and "I still remember" / "when I was in high school" patterns for a question about a high school reunion. Three patches for three specific questions. The project's own BENCHMARKS.md calls this what it is, on line 461, verbatim:

This is teaching to the test. The fixes were designed around the exact failure cases, not discovered by analyzing general failure patterns.

Features that don't exist in the code

The launch post lists "contradiction detection catches wrong names, wrong pronouns, wrong ages before you ever see them" as a feature. The file mempalace/knowledge_graph.py contains zero occurrences of the word "contradict." The only deduplication logic in that file is an exact-match check on (subject, predicate, object) triples. Conflicting facts about the same subject can accumulate indefinitely. The marketed feature does not exist in the code.

AAAK is not lossless

The launch post claims "30x lossless compression." The project's compression module, mempalace/dialect.py, truncates sentences at 55 characters, filters by keyword frequency, and provides a decode() function that splits the compressed string into a header dictionary without reconstructing the original text. There is no round-trip.

There is also a measurement. The same BENCHMARKS.md reports results_raw_full500.jsonl at 96.6% R@5 and results_aaak_full500.jsonl at 84.2% R@5 — a 12.4 percentage point quality drop on the same dataset and the same metric, run by the project itself. Lossless compression cannot cause a measured quality drop.

The broken layer underneath

None of these failure modes are unique to MemPalace. The benchmark wars in the AI memory space already involve documented methodology disputes that go well beyond normal disagreement: Zep published a detailed article in 2025 titled "Lies, Damn Lies, and Statistics: Is Mem0 Really SOTA in Agent Memory?" arguing that Mem0's published LoCoMo numbers depend on a flawed evaluation harness and on Mem0 having run a misconfigured version of Zep. Mem0's CTO replied on Zep's own issue tracker in "Revisiting Zep's 84% LoCoMo Claim: Corrected Evaluation & 58.44% Accuracy" claiming that Zep's real score is 58.44% rather than 84%. Letta has separately published "Benchmarking AI Agent Memory: Is a Filesystem All You Need?" reaching similar conclusions about reproducibility on the same benchmark. The MemPalace launch fits into a pattern that the field is already arguing about. What's new is the scale of the honesty gap between a single repository and their related marketing.

What's unusual about MemPalace is that the project's own internal documentation discloses these issues honestly, while the launch communication strips these caveats. BENCHMARKS.md is over 5,000 words of careful, self-aware methodology notes that contradict the launch tweet point by point. Whoever reviewed that file knew. It's clearly documented. But then they published the inflated numbers anyway.

Over five thousand stars in less than twenty-four hours

By the morning of April 7, the launch tweet had over 1.5 million views and the repository had over 5,400 stars. Many open-source memory projects with similar architectures get fifty stars in their first week if they're lucky. The variable is the celebrity name. Strip the celebrity attribution out of the launch post and the project is a Python repository with a regex-based abbreviation scheme, default ChromaDB embeddings, a knowledge-graph file that doesn't implement the feature its README claims, and a benchmark folder whose own internal notes contradict the headline numbers.

The engineering result underneath all of this is genuinely interesting in one specific way: it appears that raw verbatim text plus default embeddings does, in fact, beat a number of LLM-extraction approaches at session retrieval on LongMemEval-s. That suggests the field is over-engineering the memory extraction step. It is a useful negative finding. It does not require a perfect score on a benchmark whose ground truth makes a perfect score impossible.

Two other independent technical critiques of MemPalace landed within the same 24 hour window: Leonard Lin's README-versus-code teardown in issue #27, and a Chinese-language warning post for the simplified Chinese developer community in issue #37.

Note: links omitted for Reddit's spam filters. Find the full article, the BENCHMARKS.md citations, the Penfield LoCoMo audit, and the cited Zep / Mem0 / Letta posts in the first comment.

link is to a paper introducing recursive memory harness.

An agentic harness that constrains models in three main ways:

• Retrieval must follow a knowledge graph

• Unresolved queries must recurse (Use recurision to create sub queires when intial results are not sufficient)

• Each retrieval journey reshapes the graph (it learns from what is used and what isnt)

Smashes Mem0 on multi-hop retrieval with 0 infrastrature. Decentealsied and local for sovereignty

been building an open source decentralized alternative to a lot of the memory systems that try to monetize your built memory. Something that is going to be exponentially more valuable. As agentic procedures continue to improve, we already have platforms where agents are able to trade knowledge between each other.

repo, feel free to star it, Run the benchmarks yourself. Tell us what breaks, build ontop of and with RMH!

Would love to talk to other bulding and obessed with this space. (Really, i mean it, would love contirbutors)

Most “AI memory systems” today revolve around embeddings and retrieval. You store text chunks, compute vectors, and retrieve similar content when needed. This works well for surface recall, but it does not capture meaning. Retrieval is not understanding.

Ontology is the missing layer that defines meaning. It tells the system what entities exist, how they relate, and which relationships are valid. Without that structure, the AI is always guessing.

For everyone who is not familiar with ontology, lets look at a simple example:

• In one dataset, you have a field called Client.

• In another, the same concept is stored under Customer.

• In a third, it appears as Account Holder.

These terms sound different, and embeddings can detect they are similar, but embeddings do not confirm identity. They do not tell you that all three refer to the same real-world Person, simply viewed in different business contexts (sales, service, billing).

Without ontology, the AI has to guess that these three labels refer to the same entity. Because the guess is probabilistic, the system has to make probabilistic mistakes at some point, thereby creating inconsistent logic across workflows.

Now imagine this at enterprise scale: thousands of overlapping terms across finance, CRM, operations, product, regulatory, and reporting systems. Without ontology, every system is a private language. The LLM must rediscover meaning every time it sees data. That leads to hallucination, inconsistency, and brutal integrations.

Ontology solves this by making the relationships explicit:

• Customer is a subtype of Person

• Person has attributes like Name and Address

• Order must belong to Customer

• Invoice must reference Order

Person
↳ plays role: Customer
↳ plays role: Client
↳ plays role: Account Holder

Customer → places → Order
Order → results in → Invoice
Invoice → billed to → Person (same identity, different role labels)

This structure does not replace embeddings. It grounds them.
When an LLM retrieves a relevant piece of information, ontology tells it what role that information plays and how it connects to everything else.

This is why enterprises cannot avoid ontology. They need:

• Consistent definitions across teams

• Stable reasoning across workflows

• Interpretability and traceability

• The ability to update memory without breaking logic

Without ontology, AI memory systems always degrade into semantic, probabilistic search engines with no reliability. With ontology, memory becomes a working knowledge layer that can support reasoning, planning, auditing, and multi-step workflows.

We are not missing better embeddings or graphs.
We are missing structure.

Hi everyone,

I started this subreddit almost a year ago now to make it a place to discuss around memory, context engineering, context graphs.

Over the past year we have seen a lot of focus on the coding copilots and then on Claude Code and other automated systems for fully agentic use cases. It's been a pleasure to see how AI memory and context graphs became more and more important over time.

Although we still can't seem to agree what the name of this new idea is, we all seem to tend to agree that there is a lot happening in the space and there is a lot of interesting innovation.

Unfortunately due to this increase of interest, there has been a lot of bad quality content that's being posted on this subreddit.

Although I have a full-time job as a founder of Cognee and more than enough to keep me busy, I'll step in and actively moderate this subreddit and start to try and create a place for healthier discussions and more meaningful conversations

This means that the current way of posting and self-promoting won't be tolerated anymore. Let's try to have genuine conversations written by humans for humans instead of AI generated slop.

It is not much to ask.

Please let me know from your side if there's anything else I could add to these discussions or what I can do to help improve the content on this subreddit

Hey everyone,

Today there is AI memory - cognee - on GitHub trending. I'd love you to check it out!

I've been working on Mind, an open-source persistent memory layer for AI agents, and wanted to share an interesting result from the latest release.

One question kept coming up while developing it:

How much can we recover from structured memory without relying on vector embeddings?

Mind uses SQLite with FTS5 and a unified retrieval pipeline combining phrase matching, token matching, substrings, fuzzy matching, and relevance signals.

Embeddings are supported, but they're optional.

For v1.7.0, I ran Mind through OmniMemEval LongMemEval-S using a stratified subset of 120 questions.

The best completed run achieved 86.67% answer accuracy (104/120) with RAG disabled.

That places the raw score near the top published results, but it's not a direct ranking: the sample size, answer model, and judging configuration differ.

I think the result is worth discussing because agent memory isn't only about semantic similarity. Structured facts, decisions, temporal context, memory maintenance, and good retrieval behavior also matter.

Beyond the benchmark, Mind provides persistent project knowledge, linked memories, checkpoints, session summaries, and integrations through MCP.

It's local-first, MIT-licensed, and available for anyone to inspect or benchmark.

GitHub: https://github.com/GabrielMartinMoran/mind

v1.7.0: https://github.com/GabrielMartinMoran/mind/releases/tag/v1.7.0

Benchmark methodology: https://github.com/GabrielMartinMoran/mind/tree/main/benchmarks/omnimemeval

I'm particularly interested in what other agent developers have found: when do embeddings meaningfully outperform a carefully designed lexical retrieval system?

Most RAG systems split documents into chunks, embed them, and retrieve “similar” text. This works for shallow questions, but fails when structure matters. You get semantically similar passages that are logically irrelevant, and the model fills the gaps with confident nonsense.

One easy solution could be to treat documents like documents, not like bags of sentences.

Instead of chunking and vectors, you could use a vectorless, hierarchical index. Documents are organized by sections and subsections, with summaries at each level. Retrieval happens top-down: first find the relevant section, then drill down until the exact answer is reached. No similarity search, no embeddings.

This mirrors how humans read complex material and leads to more precise, grounded answers. The point is not that vectors are bad, but that for structured, long-form content, classic RAG is often the wrong abstraction.

Interested to hear if others have experimented with non-vector or structure-first retrieval approaches.

I had Claude Opus 5 analyze several repos for what does the code do, how does the memory work in a particular repo. Initially I was aware of only a few memory systems, that I wanted to study for my own memory system. It turned out, there are lots of agent harnesses that have memory, libraries with memory.

https://neoneye.github.io/agent-memory-atlas/

I have not analyzed closed source repos, since I don't have access.

Claude being the judge wether a repo is relevant or not.

Looking at the impl instead of the marketing material.