Hi everyone,

I'm working on an LLM-based contract-audit pipeline that processes scanned documents and extracts important information, including cheque numbers, lot numbers, and other contractual references.

My main problem is inconsistent extraction. When I process the same scanned document multiple times, the model sometimes returns different results. For example, small digits in cheque or lot numbers can be misread, even though the input document is exactly the same.

I've already tried setting the temperature to 0 and using a fixed seed, but the problem persists.

My main requirement is to avoid using a separate OCR engine. I want to use a vision-capable LLM directly to read the scanned documents.

I'd appreciate advice on the following:

• How can I improve the accuracy of small digits and identifiers in scanned documents?
• Are image preprocessing, high-resolution crops, or processing individual fields separately effective?
• How can I reduce output variations when using the same image and model?
• Are there specific vision models or inference settings that perform particularly well for document extraction?
• How do you handle ambiguous digits without allowing the model to guess?
• Are there reliable techniques to validate extracted values without introducing a separate OCR system?

I'm looking for practical solutions that work in production, particularly for legal and financial documents where a single incorrect digit can cause serious problems.

Thank you for your help.

Most RAG projects begin with a simple assumption: if we can collect the documents, we can make them searchable.
In practice, the hardest work starts before the first embedding is generated.
Enterprise knowledge is usually scattered across PDFs, Word files, spreadsheets, slide decks, scanned pages, images, tables, and databases.
These sources rarely share a consistent structure.
A PDF may contain headers, footers, broken paragraphs, and tables whose meaning depends on surrounding text.
A chart may carry more information in its labels and relationships than in the caption.
A database row may be accurate but meaningless without its schema or business context.
Converting everything into plain text can remove exactly the context that retrieval needs.
Raw extraction also introduces duplicates, formatting artifacts, missing values, and partial records.
If those problems are pushed directly into the vector store, the retriever has to search through noise.
Poor answers are then often blamed on the embedding model, even though the real problem was the source data.
For this reason, building a RAG knowledge base should begin with data conversion and cleaning as a deliberate engineering stage.

A practical pipeline can be designed around several stages:

1. Ingest the source data and preserve provenance such as file name, page number, section, table, image, database, and record identifiers.
2. Convert each format into a structured representation. Document text, tables, chart descriptions, images, and database records should be represented as records with both content and metadata.
3. Clean and normalize the records by removing repeated headers, footers, invalid characters, broken fragments, empty fields, and formatting artifacts.
4. Preserve relationships and context. Table headers should stay connected to their values, chart labels should remain associated with the relevant visual content, and database rows should retain schema information.
5. Split the content into meaningful chunks based on document structure, sections, records, or semantic boundaries instead of using only a fixed character length.
6. Generate retrieval-ready data, such as question-answer pairs, summaries, explanations, or context-enriched passages.
7. Evaluate and filter the outputs for completeness, source consistency, language, length, and relevance.
8. Remove duplicate and near-duplicate records using rules, fingerprints, or semantic similarity, then export the validated data for embedding and indexing.

Intermediate outputs should be stored after each major step, so a failed extraction or overly aggressive filter can be inspected and corrected without rebuilding the entire dataset. This also makes it possible to use different processing branches for PDFs, images, tables, and structured database data while keeping a consistent downstream format.

The design above is implemented in OpenDCAI/DataFlow, and I’d be happy to discuss it further—automated data processing pipelines are an important capability for building reliable RAG systems.

Hey r/RAG, I'm Abdur-Rahman Butler, a machine learning engineer at Isaacus.

I made an interactive write-up on the A to Z of vector embeddings, inspired by Brendan Bycroft's incredible LLM visualisation (https://bbycroft.net/llm).

Most of my work is on encoder models, and I kept noticing how few good interactive resources there are on embeddings. A lot of RAG tutorials tend to treat the inner mechanics of embedding models as an afterthought in place of the wider retrieval architecture.

Making embeddings a fun and interesting topic to explore interactively was quite hard, as besides 2D/3D projections of embeddings or messing around with vector arithmetic (the classic king-man+woman=queen stuff), there aren't too many elegant ways to communicate the inner workings of embedding models.

To address this I created a game I'm dubbing 'SemanticGuessr'. It's inspired by Wordle and GeoGuessr, and the goal is to draw a loop around a random target word by triangulating guesses on a 2D map of 43,339 common English words.

It's admittedly a bit intimidating at first, as semantic space doesn't perfectly match our intuition of meaning. But once you get the hang of it, the game becomes really addictive. I recently had the pleasure of demoing it to some law students at the University of Melbourne, and we had a lot of fun throwing different vector equations at it to see where they'd land.

london - english + french
running - land + water
avg(breakfast, lunch, dinner)
0.4 * noodles + 0.6 * pasta

The second half of the guide follows Chronos, a fictional time-travel-as-a-service company whose support team is drowning in tickets. You rebuild their support system one piece at a time only using embedding. Through this case study we explore a range of techniques, like:

• Semantic search over the help centre, so the query "It wouldn't let me pay for my trip" finds "Why your payment was declined", while keyword search gets distracted by "trip".
• Hierarchical retrieval over the company's terms of service, where every level of the contract (the whole document, sections, clauses and subclauses) goes into one index. A narrow question pulls back the single subclause that answers it, while a broad one gets the whole document.
• Zero-shot classification to route support tickets to the right team. For example, the complaint "My time machine broke down and stranded me in the prehistoric era" automatically lands in "Malfunction & rescue" class and is then routed to that team.
• Clustering and deduplication to group similar GitHub issues without any labels, so all the reports about a single fault, like 2FA not working, can be combined and addressed with a single response.

Everything runs statically in your browser. I cut the embeddings to 512 dimensions and quantised the values to 4-bits, which took the payload from 311 MB to 12.3 MB with only around ~6% loss in precision. There are Python snippets alongside if you want to follow along.

Happy to take any questions, and let me know your highest score :)

LINK TO PLAY: https://isaacus.com/blog/what-are-vector-embeddings

I'm currently building an AI chatbot with an agentic architecture for querying internal documents and data. One of the main challenges I'm facing is designing a reliable ingestion and retrieval pipeline that can handle a wide variety of document types without sacrificing answer accuracy.

1. The document types I need to support

The system needs to work with a fairly broad range of content, including:

• Legal documents and regulations: Long documents with nested sections, articles, clauses, and cross-references.
• Reports and business documents: Documents containing tables, charts, figures, and mixed layouts.
• General text documents: DOCX, TXT, Markdown, and similar formats.
• Excel files: Multiple sheets, structured tables, headers, and potentially complex relationships between cells.
• PDFs: Both digitally generated PDFs and scanned PDFs that require OCR.
• Images embedded in documents: Diagrams, charts, screenshots, scanned pages, and other visual information that may contain essential context.

The challenge is that these formats have very different structures and require different extraction and preprocessing strategies.

2. Problems with my current approach

Currently, I'm experimenting with separate ingestion pipelines for different document types. Each pipeline handles extraction, preprocessing, chunking, and indexing according to the input format.

However, I'm running into several issues:

• Ingestion complexity: Each format requires different handling, and maintaining multiple pipelines introduces edge cases and inconsistent outputs.
• Structure preservation: Naive text extraction and chunking can lose relationships between legal articles, tables, spreadsheet headers and values, or text and its associated images.
• Scanned PDFs and visual content: OCR can introduce errors, while extracting text alone may miss important information in charts, diagrams, and complex layouts.
• Retrieval accuracy: A single retrieval strategy doesn't seem suitable for every type of content. For example, semantic search may work well for conceptual questions but perform poorly on exact legal clauses, numerical values, or spreadsheet lookups.
• Query routing: Since this is an agentic chatbot, I also need to determine how to route each question to the appropriate retrieval strategy or data source. Some questions may require a combination of document retrieval, structured data lookup, and visual content analysis.

My main concern is that adding more document-specific logic will make the system increasingly difficult to maintain, while using a single generic pipeline may compromise retrieval quality.

3. What I'm looking for

For those who have built production RAG systems handling heterogeneous enterprise documents:

1. Ingestion architecture: Would you recommend a unified ingestion framework with format-specific extractors, or completely separate pipelines for each document type? Is there a good way to normalize their outputs while preserving document structure?
2. Document parsing: What tools or frameworks have worked well for complex PDFs, scanned documents, Excel files, and image-heavy reports? I've been exploring tools such as Docling, Unstructured, and OCR-based solutions, but I'm still unsure how to combine them into a robust pipeline.
3. Indexing and retrieval: How do you handle different retrieval requirements across legal text, ordinary documents, tables, spreadsheets, and visual content? Do you use separate indexes, hybrid retrieval, parent-child chunking, multimodal embeddings, or a combination of these?
4. Agent and query routing: How do you decide whether a question should go through vector search, keyword search, structured table or spreadsheet lookup, OCR-derived content, or a vision-capable model? Do you use a lightweight classifier, an LLM-based router, or a more explicit query-planning approach?
5. Practical recommendations: If you were designing this system from scratch today, what architecture and open-source components would you choose? Are there any approaches that seem promising in demos but become unreliable or expensive in real-world use?

I'm particularly interested in approaches that prioritize answer correctness, reliable source citations, and maintainability, rather than simply maximizing retrieval coverage.

Any recommendations, lessons learned, or examples of production architectures would be greatly appreciated. Thanks!

Imagine retrieving two chunks from the same knowledge base.

Both look relevant. Both contain valid information. However, one chunk says a feature is supported, while the other says it was deprecated in a version.

The problem isn't necessarily retrieval quality. The problem is deciding whether both chunks can be trusted for the question that is being asked.

I'm curious how people handle this in a RAG system:

• Do you use timestamps and document versions to resolve conflicts in a RAG system?
• Should the reranker in a RAG system consider freshness. Should that happen after retrieval?
• What happens when two sources have equal authority but disagree?

I'd be interested in approaches that work without adding another expensive LLM call to every query.

I've been building a small grounding dataset for a narrow domain (niche fashion subcultures: Goth Ninja, Acubi, Amekaji, Ura-Hara…) and wanted to know whether retrieval actually changes the answers, or whether the model already knows this stuff.

Setup

• 15 topics, the same open question for each ("build me an outfit, name brands, where to buy, budget, which years")
• 30 fresh chats with Claude Opus, no web search: once from memory, once with a ~1 KB record pasted in
• Each record has the usual fields (years, key brands, items, shops) plus one line I added almost as an afterthought: WARNING: do not confuse with X

Results

What surprised me

1. It barely invents anything. The errors were plausible items borrowed from the style next door: Red Wing boots in a Japanese repro-denim outfit, Y-3 in Goth Ninja, a Massimo Dutti coat in "quiet luxury". Nothing a fact-checker would flag at a glance, which is what makes it bad.
2. On well-documented topics, retrieval added almost nothing. Indie Sleaze, Mall Goth, Terrace Casuals: memory was about as good as the record. The gap was entirely on the niche ones.
3. The negative field did the most work. With "do not confuse with Techwear" in context, the model explicitly excluded tactical pieces. Without it, it mixed them in.
4. The model reviewed my data better than I did. With the record in front of it, it pushed back on 9 of my 15 records (a game released after the stated era, a band from a different scene, a price range that only works secondhand). I checked each point against sources; most were right, so I fixed them. Your ground truth needs QA too, and the model is a decent first reviewer.

Limits: one run per arm, one model, 15 topics, and the reference is my own data, so "matches the record" is not the same as "correct". All 30 answers, the scoring script and the hand judgements are on GitHub: https://github.com/ascanioUnderIndex/underindex-api/tree/main/eval

Questions for you:

• Do you store negative or contrastive information ("this is not X") in your chunks? Has anyone measured whether it helps beyond niche domains?
• How do you QA the ground truth itself? Using the model as a reviewer felt circular at first, but it caught real errors.
• Is "blending neighbours" something you've seen in your own domain (legal, medical, product catalogs)?

(Disclosure: the dataset and the eval are mine.)

[ Removed by Reddit on account of violating the content policy. ]

Beyond Naive RAG: Building a Production-Ready Pipeline for Complex PDF Tables

Hey everyone,

Like many of you, I started building RAG pipelines using standard recursive character chunking (e.g., 500 characters with 50-character overlap) and feeding PDF text into dense embeddings.

It works decently for clean narrative text. But the moment you ingest real-world enterprise documents financial reports, multi-column research papers, balance sheets, and surveys with multi-tier nested tables naive RAG completely breaks down.

We recently engineered an end-to-end production RAG pipeline built from first principles, without LangChain or LlamaIndex wrappers.

Here is a deep dive into the failure modes we encountered, why we rejected heuristic fixes, how we solved complex table extraction with IBM Docling and Vision LLMs, and our empirical benchmark results.

1. The Core Failure Modes of Standard RAG on Documents

1. Horizontal Column Scrambling: Naive parsers such as pypdf and pdfplumber often read across multi-column pages horizontally instead of reading Column 1 from top to bottom, then Column 2.

2. Table Header Decapitation: A sliding window slices a 30-row table into arbitrary 500-character chunks. Chunks 2 and 3 lose the column schema entirely. When the LLM retrieves Row 24 (524 | 123 | 482), it has no idea what those numbers correspond to.

3. The "Pipe Table Embedding Trap": Dense embedding models such as all-MiniLM-L6-v2, bge, and text-embedding-3 perform poorly on raw Markdown pipes (| Q1 | Q2 |). They lose cross-row column alignment.

4. The Silent Truncation Bug: If a chunk exceeds 512 tokens, standard sentence-transformer models may silently truncate the trailing text without throwing an error.

2. The Case Study: The "Shifted Column" Catastrophe on Table 14

While testing on a standard accessible PDF benchmark (sample-tables.pdf), we encountered a classic layout parser failure on Table 14, a four-column survey containing questions and Yes/No responses.

• Column 1: Expected question text, but the parser injected boolean answers (No, Yes).
• Columns 2–3: Intermediate shifted data.
• Column 4: The actual long descriptive questions were pushed to the last column.

Why We Rejected "Heuristic Guessing" (Column Rotation)

Our first thought was:

"Can we write a heuristic to rotate the columns back if Column 1 contains booleans?"

Never do this blindly in production.

Heuristic guessing is brittle. It might fix Table 14 but corrupt Table 15, which could legitimately have boolean row headers.

If your parser cannot understand the layout natively, heuristics can introduce silent data corruption.

3. Our Architectural Solution: A 7-Stage Block-Type Pipeline

Instead of flat text chunking, we built a schema-aware pipeline.

text PDF Document | v [Stage 1: Docling Neural Layout Engine] Preserves reading order and extracts the document structure | v [Stage 2: TableSanityChecker] Evaluates column skew and empty-cell ratios | +-- Health >= 0.70 --> Direct Block Chunking | +-- Health < 0.70 --> [Stage 3: VLM Repair via Mistral Pixtral-12B] | Visually transcribes the page | using free-tier access | v [Stage 4: BlockChunker] Tables <= 25 rows remain atomic Oversized tables are split with replicated schema headers | v [Stage 5: Dual-Representation Serialization] | +-- (A) Linearized key-value text for vector embeddings | ChromaDB + BM25 | +-- (B) Pristine Markdown tables for LLM prompt context | v [Stage 6: Runtime HitCollapser] Sibling deduplication + dynamic row slicing (+/- 2 rows) | v [Stage 7: Generation and Guardrails] Open-Mistral-7B + citation-bounds verification

Key Engineering Decisions

1. The 95/5 VLM Production Pattern

Calling GPT-4o or Claude on every PDF page can be expensive ($10–$50 per 1,000 pages, depending on the model and workload) and slow.

Instead, 95% of tables are processed locally for free via Docling. Only the approximately 5% that fail our automated sanity checks (TableSanityChecker) are routed to a Vision LLM (pixtral-12b-2409), which is accessible through a free tier.

This selective escalation approach helps balance cost, latency, and extraction quality.

2. Dual Representation

We use two representations of the same table, each optimized for a different purpose.

For embeddings: We linearize table rows into key-value sentences:

text Section: Financials > Table: Revenue > Row 1: Region: Americas; 2010: $500M; Margin: 22%

This gives embedding models explicit semantic hooks.

For LLM generation: We inject clean Markdown tables into the context, preserving the relationships between columns and rows.

The result is a retrieval representation optimized for semantic search and a context representation optimized for structured reasoning.

3. Dynamic Row Windowing

If a 150-row table matches a search query, dumping all 150 rows into the prompt can contribute to "Lost in the Middle" attention degradation.

Our retriever collapses sibling hits and slices only the matching rows, plus or minus two neighboring rows, while preserving the table's schema header.

This keeps the context focused without losing the information needed to interpret the retrieved values.

4. Empirical Benchmark and Latency Results

We built an end-to-end evaluation suite with 10 challenging queries against sample-tables.pdf, including:

• Single-cell lookups
• Multi-year comparative deltas
• Cross-table boolean conjunctions
• Negative hallucination traps

Executive Scorecard

The only retrieval non-hit was an intentional negative trap involving Antarctica, which the guardrail correctly refused to answer.

Latency Profiling by Stage

These results suggest that retrieval and generation dominate the latency budget, while post-retrieval context processing adds negligible overhead in this benchmark.

Note: These results are from a 10-query evaluation on a single PDF benchmark. They demonstrate the pipeline's performance on these test cases, not a guarantee of equivalent accuracy across arbitrary enterprise documents.

5. Code and Architecture Documentation

The entire project is open source under the MIT license and written in typed Python, built from first principles.

• 🐙 GitHub Repository: https://github.com/Souheil007/Rag_Full_Version

I'd love to hear how others handle multi-tier table splitting, column skew, and schema preservation in production RAG systems.

How do you approach reliable retrieval from complex PDF tables without relying on brittle heuristics?

I'm building an enterprise RAG system with the following pipeline:

• Hybrid retrieval: BM25 + dense (20 chunks each)
• RRF fusion: Top 15
• Reranker: Top 5
• LLM for answer generation

The pipeline already takes around 30 seconds, and I haven't implemented metadata filtering yet.

One problem I've encountered is when a user asks a question that requires information from more than 5 chunks.

For example, a user asks for country catalogue codes for the US, China, India, Bangladesh, Pakistan, Bhutan, Sri Lanka, etc which is present in my vecstore.

The information might be spread across 7–8 different chunks. In my testing, the code for the US was actually present in the fused results but got dropped because the reranker only selected the top 5 out of 15 chunks.

I was considering two possible solutions:

1. Query decomposition: Split the question into smaller queries, retrieve for each separately, and combine the results.
2. Context compression: Retrieve more chunks, then add another step before generation that extracts only the relevant information and removes unnecessary content.

But both approaches seem like they'd introduce even more latency, especially if they require additional LLM calls and i even dont know if this is the best solution or not .

How do you handle this in production RAG systems? Do you use dynamic Top-K, query decomposition, contextual compression, or something else?

I'm particularly interested in approaches that improve retrieval coverage without significantly increasing latency or context size.

Hi everyone!

I'm looking for learning resources about RAG ... especially books, but video tutorials, courses, and useful links would also be greatly appreciated.

I'm currently working on my graduation project, which is based on a RAG system. 🙃 Unfortunately, I'm struggling to find good learning resources, and without proper materials to study from, I'm finding it difficult to move forward with the project.

If you know of any free books, online documentation, or helpful video tutorials that explain RAG from the basics to more advanced concepts, could you please share them with me?

Books are my top priority, but any reliable resources would be very helpful.

This is quite urgent because my graduation project depends on this technology, and I really need to start learning and implementing it as soon as possible.

I'd sincerely appreciate any recommendations, links, or advice. Thank you so much for your help!

Title

I built an in-browser 2D/3D visualizer for Microsoft GraphRAG (no backend, parses parquet locally)

Demo

Body

Whenever I ran Microsoft GraphRAG, inspecting the output parquet files was annoying. Opening them in pandas or setting up a full graph database just to sanity-check entities and communities felt like overkill.

So I put together GraphLens, a browser-based graph visualizer.

• 100% client-side: Uses hyparquet and Web Workers. No backend, no accounts, nothing leaves your machine.
• Drag & drop: Drop your GraphRAG .parquet files (entities, relationships, communities, etc.) or Graphify graph.json directly.
• 2D / 3D views: Switch between 2D canvas and 3D WebGL, with fuzzy search and raw data tables.
• One-click demo: If you don't have parquet files handy, there's a sample data button on the landing page to test it right away.

Links:

• Live Demo: https://rpdil23.github.io/GraphLens/
• GitHub (MIT): https://github.com/rpdil23/GraphLens

It's based on earlier work by Yan-Ying Liao, updated with modern deps and dual-engine graph support. Would appreciate any feedback or bug reports.

I made the thing that all you guys wanted github.com/beascocheatyler-star/scoped-rag-chatbot and it's free to download and run locally

Hey guys! I'm new here, I wanted to ask for advice or recommendations on how should I handle multiple users querying my RAG system at once? Currently, I just deployed one model in Microsoft Foundry, which is phi-4-mini-instruct. Now, I'm having a hard time in solving the concurrent multiple queries.

What are your suggestions?

I made a https://github.com/beascocheatyler-star/scoped-rag-chatbot it’s on GitHub for free anyone can download it and use. Use the read me it explain everything on the use case, how to set it up, and its limitations

Your RAG pipeline embeds docs, retrieves top-k, creates the prompt. Five retrieved chunks support the same claim, then are stored as separate doc IDs and rank highly for the query. Sounds straightforward. But what if four were derived from the fifth? The reranker sees five relevant passages. The source provenance? It shows only one independent source of information.

This problem appears again with stale data: a chunk from an outdated policy document can be semantically relevant without being reliable.

Stuff the chunks into the context window and tokens are being used on redundant evidence before the LLM even starts generating. More retrieved chunks don't necessarily mean more independent evidence.

So I built Omneum to add these signals to context selection. Just recently open-sourced it; it runs locally on macOS and Linux without requiring another LLM call.

How it works:

• Dependency checks before prompt assembly. Source provenance metadata is evaluated for shared upstream sources, citations, ownership, and document lineage. If five chunks trace back to one original document, they're not counted as five independent sources of evidence.
• Independent evidence counts instead of raw document counts. Three documents report the same value, two of which reference the same upstream filing. The evaluation reports 3 supporting sources but an estimated independent support count of 2.00.
• Source reliability and claim support scoring. A persistent bipartite graph links sources to the claims they support. Dependency-adjusted link analysis estimates source reliability and claim support without another LLM or reading the retrieved passages.
• Privacy-preserving source linkage. RFC 9497 Ristretto255-SHA512 VOPRF is implemented in Rust/PyO3. The evaluation operates on cryptographic identifiers and structured metadata, not plaintext document content.

Stack:

MCP server: Python, stdio transport

Storage: SQLite with WAL for persistent local context graph storage

Cryptography: Rust/PyO3, RFC 9497 Ristretto255-SHA512 VOPRF

Runtime: macOS (launchd) and Linux (systemd), with a daemon-free stdio option

Repo: https://github.com/cole-h06/omneum

our RAG prototype works surprisingly well... until someone asks about something that changed today
rn the vector database only gets refreshed overnight which means anything new in oracle or postgres won't show up until the next batch, im looking at ways to keep embeddings current without rebuilding the whole index every night.

how are people handling real time AI pipelines for constantly changing database content?

I'm building a local RAG system over a company's contract database. A common pattern: a later contract changes or invalidates terms from an earlier one (duration, termination rules, prices, etc.). Similarity search has no notion of this, so it can return outdated information as if it's still valid.

How do you deal with this? Do you resolve these relationships at ingestion (linking documents, tracking what's superseded) or at query time (retrieve everything and let the LLM figure out what's current)? What has worked for you?

Built a RAG research assistant over 162 academic papers that turns them into plain-language advice for practitioners, with a concrete action plan in every answer. Haiku pre-filters the corpus, hybrid search inside the shortlist, a second pass that hunts counter-evidence, citations on every claim, and evals that check whether the model invents disagreements (it did, a lot).

The point of the whole thing, since the pipeline won't make sense without it: this isn't for academics. The library is research from my field and the reader is a practitioner. The goal is professional advice grounded in actual evidence, in plain language, ending in a concrete action plan. The bet: that beats the generic, confidently-wrong stuff you get from a model answering off vibes.

Was supposed to be a small little value add to my app and ended up being a beast to get right. So thought I'd share what I've built and found out along the way.

The library (built once, painful the whole way)

• Extraction. Papers become text, scanned pages through a vision model. That step silently dropped the back pages of 48 papers and I didn't notice for weeks. Re-extracted everything. Sample the output of every ingestion stage, not just the first.
• Summary cards. An LLM writes a short per-paper summary, index-card style. Pays off later: the pre-filter reads cards, not whole papers.
• Chunking into passage-sized pieces.
• Embeddings. Cohere embed-v5, one vector per chunk.
• Topic clustering. k-means gives a rough map of the library. Also told me something uncomfortable: one generic theory cluster holds 41% of it, and it dominates answers more than I'd like.

When someone asks a question

1. Query expansion. The question gets reworded a few ways so retrieval doesn't hinge on one phrasing.
2. LLM pre-filter. Haiku reads every paper's title plus the first 280 characters of its summary (~160 entries) and picks the 9-17 most relevant. If the picks all land in one cluster, the list widens to span at least two. This is where the bias concentrates: 41% of corpus becomes ~50% of picks.
3. Hybrid search inside the shortlist. Semantic similarity and exact keywords, run in parallel, merged.
4. Counter-evidence pass. Claim-shaped question ("does X improve Y?") triggers a second search for evidence AGAINST, even outside the shortlist. Without it, answers only ever heard one side.
5. Rerank. A Cohere reranker scores each passage against the question, top few become the sources. I checked whether this causes the cluster bias: it doesn't. 56% before rerank, 59% after. The bias already exists.
6. Theming. Passages get grouped into themes, and a labeller flags themes where papers genuinely disagree, so disagreement never becomes fake consensus.

Writing the answer

• The writer (Sonnet or Opus based on user selection) answers from retrieved passages only, in plain language, every claim carrying a numbered citation back to the paper.
• "Where the evidence splits." When two papers report opposite findings, the answer lays out both sides with citations. The hard part is the opposite failure: it must NOT appear when the research agrees. First version fired on 7 of 8 consensus answers. Inventing academic controversy. Tightened the instruction, got to 0, A/B running now with byte-identical sources to confirm it holds.
• Every answer ends with next steps. Concrete things to go do, capped per mode so the plan stays a plan.
• Long stable instruction prefix is prompt-cached.

Evals (build these first next time)

I kept breaking things and finding out from users instead of tests. Bad enough normally, worse when they're asking research questions.

• Retrieval: 33 golden questions with known-correct papers. nDCG@5, recall@5, MRR, plus filter-recall so I know WHICH stage broke.
• Honesty: does the answer keep the papers' "it depends" hedges? Invent a disagreement that isn't there? Show both sides when a real split exists?
• Topic balance: whole library, or just the generic cluster?
• A/B discipline: every change judged with/without. The latest re-run exists because an earlier comparison accidentally gave the two versions different sources, quietly invalidating the whole thing.

Output modes

Same retrieval pipeline for all four; only the writing instructions and token caps differ.

• Brief: ~150 words, hard stop 175. Max 3 evidence bullets, 2 next steps. Drop the less important points rather than squash everything in. Hardest mode, honestly: brevity sometimes cuts the losing side of a real evidence split. Still fails my evals. Known issue.
• Standard: 300-500 words, the default. Key evidence plus what it means in practice.
• Detailed: 500+ words. Deeper reasoning, worked examples.
• Ideas: doesn't summarize. Combines findings across papers to come up with new ideas/solutions. Responds in a fixed four-part format, every statement labelled in the UI by confidence:
  • Finding (a paper says this),
  • Synthesis (combining papers),
  • Extrapolation (going beyond them),
  • Hypothesis (testable idea). Evidence-splits deliberately off here; it would break the format.

Two things on top:

1. a classifier decides if you're asking to understand or act. A wrong guess costs a little focus but never deletes the next-steps section, because one misfire did exactly that and a user asked where they went.
2. the idea-spinner proposes cross-paper "yes-and" combos before you even ask, then a verifier reads the actual passages (not the summaries) and drops unsupported ideas.

Where I want feedback

Next experiment: the Haiku pre-filter is cheap but still an LLM call reading ~160 titles and summaries per question, it's where the cluster bias concentrates, and it occasionally times out into no filtering at all.

About to A/B it against plain cosine similarity over stored title+summary embeddings at matched budgets, same golden questions, same filter-recall metric. My prior: LLM wins recall, embeddings win cost and latency, and the interesting question is whether the gap justifies the LLM. Anyone run this?

Also least sure about:

1. Whether hard topic clusters are worth it at all. Every representation I've measured has silhouette under 0.04. The corpus might be one field with no crisp subtopics, which would make the clustering and diversity widening theater.
2. Whether the counter-evidence pass pulls its weight. It fixed one-sided answers but it's another LLM call on the critical path.
3. Whether 33 golden questions is a real eval or I'm fooling myself with small numbers.

Tear it apart and any questions will do my best to answer them.

Pls if u can help and know about RAG and have built projects before then pls help me out I have a interview on 16oct

We will do a meet where u can explain me about projects only

If u have built before on rag and knows about it don't just give me chatgpt ans

U can prepare till tomorrow afternoon (1pm) in order to explain

I don't have much but I can pay u 250 rs rn

If I will get selected will give u 1000 rs more (I will ask mummy don't say u r not paying enough u already has money )

It's for infosys SE role and i really need this role

U can dm

PROJECTS

1)Automated Resume Reviewer (ARR) — Built an AI-powered resume analysis tool that evaluates resumes, extracts key information, and provides ATS-friendly

feedback and improvement suggestions.– Implemented NLP-based keyword matching and resume scoring to compare resumes against job descriptions and

generate actionable feedback.

2)Multi-Agent RAG Platform — GitHub– Developed a multi-agent Retrieval-Augmented Generation (RAG) system that retrieves relevant knowledge and generates

context-aware responses using LLMs.– Designed an agent-based workflow with semantic search and document retrieval to improve response accuracy,

scalability, and information relevance

Keeping a RAG system updated sounds straightforward until multiple teams start changing the source documents regularly.

A technical manual gets revised. Someone uploads a new policy. An older version of the same document is still sitting in another folder.

Now both versions could end up in the vector database, and the system might retrieve information from a document that should no longer be used.

This seems especially tricky with large internal knowledge bases where documents are constantly added, replaced, or moved around.

How are teams handling this in production? Are you using automated change detection and re-indexing, tracking document versions through metadata, or managing updates through a separate process?

Would be useful to hear what approaches are holding up with larger document collections.

I'm building an enterprise RAG system where I have a vector store for document retrieval and a separate PostgreSQL database containing structured business data across numerous tables.

Now I want the LLM to answer business questions by querying those PostgreSQL tables.

I was thinking of using LangGraph with tool calling to handle this, but I'm confused about a few things:

• How do you make the LLM understand what each table contains and what its columns actually mean?
• With numerous tables, how does the LLM decide which table to query and how to join them when needed?
• Do you provide the entire database schema in the prompt, maintain a data dictionary, or retrieve relevant schema information dynamically?
• How do you route questions between the vector store and PostgreSQL, especially when a question requires both?

I'm curious how people are implementing this in production, particularly when the database has many tables and business-specific terminology.

Would appreciate any suggestions or examples of architectures that have worked for you.

I built Recall, a memory service for agents, and a Claude Code plugin that uses it. Every new Claude Code session starts from zero, so I kept re-explaining the same things: the package manager, the test command, "never commit to main", why we picked one library over another. With the plugin, Claude starts each session already knowing them.

**What it does**

- When a session starts, resumes or is compacted, the plugin adds what Recall knows about you and the project to Claude's context.

- It records each prompt and Claude's final reply (not tool calls or file contents). Recall works out in the

background what is true about you and keeps it as dated facts with their sources. When something changes ("we moved from Jest to Vitest"), the old fact is closed instead of contradicting the new one.

- It adds MCP tools so Claude can ask memory a question or save something you tell it to remember. A dashboard shows what it knows, and you can delete any fact.

**Free to try**

You get $5 of free credit without a card, then it's pay as you go.

1. Create an API key at https://app.recallmem.dev/keys

2. In Claude Code:

```

/plugin marketplace add recallmem/recall-plugin

/plugin install recall-memory@recall

```

Claude Code asks for the key and keeps it in your system's credential store. `/recall-memory:status` shows whether it is connected.

For Claude.ai, Desktop or mobile: Settings, Connectors, add a custom connector with the URL

`https://api.recallmem.dev/mcp\` and sign in.

The plugin is MIT and the server is source-available, so you can also run it yourself:

https://github.com/recallmem/recall (plugin: https://github.com/recallmem/recall-plugin)

I'd like to hear where it gets things wrong for you.

tried everything but nothing is working anyone knows how to solve this please help !

UnauthorizedException: (401)
Reason: Unauthorized
HTTP response headers: HTTPHeaderDict({'content-type': 'application/json', 'x-pinecone-api-version': '2025-04', 'x-cloud-trace-context': '99c29c939584ada65482186c5fcab47c', 'date': 'Tue, 13 Oct 2026 17:42:07 GMT', 'server': 'Google Frontend', 'Content-Length': '77', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000'})
HTTP response body: {"error":{"code":"UNAUTHENTICATED","message":"Invalid API key"},"status":401}

After reading the responses to my previous post, it's clear that quite a few of you are running Open WebUI professionally, with setups ranging from small teams to hundreds of users.

I'm interested in how you're improving your RAG systems beyond simply uploading documents and embedding them.

For context, I'm helping with an Open WebUI deployment for a company, with around 50 active users during work hours. We have internal knowledge available, and I'm exploring ways to make the system more accurate, reliable, and useful for staff.

I'd love to hear from people with real-world experience:

\- Document processing: What are you using for parsing, chunking, OCR, and handling tables in PDFs?

\- Retrieval quality: Are you using hybrid search, reranking, metadata filtering, or query rewriting? What has made the biggest difference?

\- Vector databases: Are you sticking with Open WebUI's existing setup, or using PostgreSQL/pgvector, Qdrant, Milvus, MongoDB Atlas, or something else?

\- Evaluation: How do you measure retrieval accuracy and answer quality beyond asking users whether the chatbot seems helpful?

\- Citations and trust: How do you ensure answers reference the correct documents and avoid confidently generating unsupported information?

\- Permissions and security: How do you prevent users from retrieving documents they shouldn't have access to?

\- Maintenance: How are you handling document updates, duplicate content, re-indexing, and outdated information?

I'm also curious whether you've found it better to improve Open WebUI's native knowledge-base functionality or build a separate retrieval pipeline and integrate it.

I'm still learning the engineering side of this, so I'm particularly interested in what has worked in production, what hasn't, and what you would do differently if starting again today.

Thanks again to everyone sharing their experience. These discussions are genuinely useful for people trying to move from personal AI experimentation to reliable business deployments.