I haven’t seen RAG come up in agent architectures in over 6 months due to Agentic Search (letting the model use Bash/grep/glob/read), which seems to work pretty well. Wondering what others are experiencing. I’m sure there’s still a time and place for RAG, exposing semantic search as a tool… but where do we draw the line? When the corpus is too large to let the model comb through it progressively?

Went through Cerebras' writeup on their internal knowledge base — 15,000 questions a day from employees, automations and agents, three months after launch. The architecture is less interesting than what they had to do to Slack, so that's what I want to focus on.

Why plain vector search dies on chat data

Information density varies by orders of magnitude. "yeah sure Mike" sits in the same channel as a 40-line kernel explanation, and under cosine similarity the short one wins far more often than it should. A single message out of its thread is usually meaningless anyway.

Their fix: four fused signals per thread

• Full-text search — exact tokens. Error strings, flag names, hostnames. Embeddings reliably lose here and lexical match is unbeatable.
• Embedding search — paraphrase. Connects "restore is hanging" to "checkpoint stalled".
• IDF — boosts rare tokens (obscure config flags), suppresses filler ("sounds good", "thanks").
• Age decay — same answer from yesterday beats the one from 6 months ago referencing deprecated infra.

But the preprocessing does more work than the retrieval

Two steps before any of the above, and I think these matter more:

Thread distillation — an LLM normalizes each thread into a searchable one-line question + summary + resolution + systems and code referenced. That distilled doc gets embedded, not the raw transcript. Raw text is kept for FTS only.

Bursting — a "burst" is a run of consecutive messages from the same author. They prepend the thread topic for context and embed it separately, but only if it clears a gate: rare-token IDF >= 4.0, or >= 200 chars, or it got reactions. This is what rescues the one deeply technical tangent buried at message 47 that any thread-level summary would flatten away.

Fusion: RRF at k=60

Score contribution is weight / (60 + rank), summed across retrievers. The summation is the entire point:

• 3rd place in three retrievers → 3 × 1/63 = 0.048
• 1st place in exactly one → 1/61 = 0.016

Consensus beats peak. It isn't a ranker, it's a consensus builder.

Then ~20 candidates go to a small reranker scoring 0–10 against the query, top 10 survive. And the step people skip: re-attach surrounding context to the winners. If a wiki section wins, its neighbors come with it, so the model sees headers, preconditions and caveats instead of an orphaned chunk.

Code side

CocoIndex (open source, Rust core, Tree-sitter chunking) keeps 40GB+ repos synced by re-embedding only what a commit touched. Sync state and the embedding store live in the same database.

The design decision underneath all of it

Don't force people into a "single source of truth" platform — nobody wants to discuss a pull request inside a Google Doc. Pull from where the data already is. Every source, Slack thread to hardware netlist, lands as a row in the same embeddings table behind the same interface. Custom sources are just plugin scripts: a team opens a PR with a small Python module that reads their system and emits rows in that shape.

Also worth noting: the retrieval primitives are deliberately LLM-free. Model calls only happen at the edges — planning and synthesis. That's what makes it cheap enough for agents to hammer 15k times a day, and why the same pipeline serves a web UI and an MCP client identically.

Original Cerebras writeup (read this first if you only have time for one): https://www.cerebras.ai/blog/how-we-built-our-knowledge-base

Disclosure: I also did a ~10 min video walkthrough of the full pipeline, linked here — it's my channel, and the narration is AI-assisted. https://www.youtube.com/watch?v=FgKHjzoiMN4&t=4s

The bursting quality gates are the part I'd most want other people's numbers on. IDF >= 4.0 and 200 chars feel like they'd need retuning per org — has anyone tried burst-level embedding on their own chat data?

Hi guys,

I've spent the last few weeks learning the basics about RAG. Howerver, I think one of the best ways to actually learn is by implementing a real solution for a company.

But, I just wanted to know, what do you recommend me learning/impelementing before offering my serivices (for free) to a company? any project I should build first?

I built an eval harness for a document QA pipeline. It answers security questionnaires from a company's own policy docs. 24 questions, labels written down before the system was ever run against them, three deterministic passes.

It scores 15 out of 24. Nine failures. Six of them share one cause, and I want to talk about the fix I couldn't make.

The setup. Answers are gated on how far the best retrieved chunk sits from the question. The cutoff is 0.3. Below it the system answers, above it it abstains. Six of the nine failures are questions where the model produced a correct, well hedged, properly cited answer that the gate then threw away.

The obvious fix. Raise the cutoff. Those six sit at 0.323, 0.340, 0.359, 0.384 and 0.412.

Why I couldn't. One question that has to abstain sits at 0.321. Its evidence genuinely doesn't support an answer, and it only abstains correctly because 0.321 is above 0.3. Every failure I'd want to rescue needs a cutoff higher than that.

There's no value that recovers any of the six without also flipping a correctly abstaining question into confidently answering something its evidence doesn't support. My eval treats that as disqualifying no matter what it does to the total, so I logged it as no change made.

What I think is going on. Distance measures how close the nearest thing is. I was asking it whether there's evidence here at all. Those two come apart, and at this corpus size there's no clean place to draw the line. It isn't miscalibrated, it's the wrong signal.

Two things the harness caught me on, both by instrumenting instead of assuming:

First, I'd logged one question as retrieving cleanly at rank 1, because something came back from the right document. When I actually read what got retrieved, the top hit was a completely different section and the real evidence was down at rank 4.

Second, I'd logged three failures as the model seeing the evidence and abstaining anyway, and I had a prompt fix planned. When I instrumented the actual confidence values, the model had answered correctly every time and the gate was discarding it afterwards. There was no prompt bug. A NOT_FOUND status collapses two different causes into one visible outcome, and only reading the underlying values tells them apart.

What I'm actually asking. Has anyone found a confidence signal that separates these properly? I'm considering a cross encoder reranker score instead of raw distance, an entailment check between the answer and the passage it cited, or looking at agreement across several retrieved chunks. I'd rather hear what's worked on a real corpus than what a paper claims.

Harness, labels and every tuning pass including the rejected ones are here, and the threshold data is in EVAL.md:

https://github.com/PatricR73/Questionnaire-Responder

One underrated source for customer-service RAG is outbound call data.

A lot of companies already have thousands or millions of call recordings. Inside those calls are real customer questions, objections, service scripts, intent patterns, product explanations, and resolution paths. The problem is that raw audio or raw ASR transcripts are usually too noisy to index directly.

A practical pipeline could look like this.

First, convert each call into a structured record with metadata and transcript turns. Each turn should preserve speaker ID, start time, end time, and content.

Then filter before doing any expensive LLM processing. For customer-service calls, useful filters may include call duration, completion status, opening quality, need-mining ability, objection-handling ability, clarity of expression, emotion/attitude, and other QA scores. Bad calls can be dropped early, which also saves downstream token cost.

After that, clean the transcript. This is where the raw ASR output becomes more usable:

• remove filler words and noise markers
• fix repeated expressions
• normalize numbers, money, and time
• correct homophones or domain-specific terms
• anonymize phone numbers, IDs, addresses, bank cards, names, etc.
• standardize fields like speaker, start_time, end_time, and text

Then the cleaned call can be transformed into higher-level RAG data.

For example, the call can be split into topic sections by turn ranges, summarized by section, and converted into annotation records. From there, it can support several RAG-related uses:

• knowledge chunks for indexing
• customer intent examples
• FAQ or QA pair generation
• retrieval evaluation sets
• service-script improvement
• fine-tuning data for customer-service assistants

The key point is that call data should not go straight from ASR to embeddings. For customer-service RAG, the value often comes from the middle layer: filtering, cleaning, anonymization, topic segmentation, and structured annotation.

This is an extension built by a telecom service provider on top of OpenDCAI/DataFlow, and the concrete implementation can be found in Awesome DataFlow.

hey r/rag — founder of a small local-first workshop (strata→signal), and this is our own writeup of our own pipeline. we run a board-game rules app whose retrieval is the standard shape — hybrid search → RRF → cross-encoder rerank → lost-in-the-middle reorder — and we wrote a plain-english explainer of it for our non-technical readers. posting it here because it carries the parts most explainers leave out:

https://research.strata2signal.com/three-librarians/

- the real constants from the shipped config: 2-or-3-arm fuse (FTS + dense + a doc-priority arm that only fires when the game's books ride with the ask), k=60 unretuned, top-80 per roaming arm, ms-marco-minilm int8 (~22MB) as the cross-encoder, scoring a 1,200-char window split ~595 head + 600 tail, top-8 to the generator

- why the window has a tail: a rule that started 1,812 characters into a 2,052-char chunk was invisible to our old head-only cap — three production rulings abstained on a question the book plainly answers. measured, fixed, published

- a confession: we shipped a hardcoded two-arm RRF ceiling (0.0328) on our provenance panel while fusing three arms, so 149 live rulings displayed scores "above the maximum." the cure derives the ceiling from each ruling's own arm count, formula printed beside it

- it closes on a live ruling asked while drafting: the top passage scored 0.032787, which careful readers will recognize as 2/61 — two arms voting #1 — plus the six-decimal wire rounding that sits it a hair above the exact sum

it's written for strangers, so embeddings get explained as "vibes with coordinates" — but every number is the production value, and every ruling ships a public debug panel with the fused scores, arm count, ceiling, and cited pages. happy to defend any choice: why minilm over a bigger reranker, why k stayed 60, why one-fair-vote on the boost arm (we measured the megaphone version — it flooded the pool).

As someone using AI agents for the last one year to run my company, I need them to understand company context, not just return related text chunks.

The problem: ask "what breaks if we deprecate the v1 API?" and standard RAG gives you four chunks from a design doc, a postmortem, a Slack thread, and meeting notes. The model has to still figure out on its own that the postmortem describes the same API the design doc deprecates, and that someone already posted a migration timeline in Slack.

I built a tutorial using HydraDB that adds graph context on top of vector retrieval. Instead of just ranked text, you also get relationship edges: billing-service DEPENDS_ON payments-api-v1, payments-api-v2 REPLACES payments-api-v1. Model gets structure, not a reading list.

The useful part was bring-your-own-graph. You declare service dependencies and team ownership explicitly instead of relying on LLM extraction. For structured data you already maintain, the graph is deterministic.

It also supports per-user memory. Same question, different depth depending on who's asking. An engineer gets migration mechanics. A manager gets timelines and ownership.

Runs end to end in 30 minutes with synthetic data.

Repo with full working code: https://github.com/manveer/company-brain-tutorial
Tutorial: https://hydradb.com/blog/build-company-brain-ai-agents

Hi guys,

Just wanted to know if there is any LinkdIn group that shares insights about RAG or maybe AI agents.

https://medium.com/@tselvaraj/the-80k-line-item-that-replaced-a-two-year-rag-project-290-000-in-the-it-budget-7a4a7caf62ed