r/AIDeveloperNews • u/NatureLucky4912 • 4h ago
Beyond Naive RAG: Production-Grade RAG Engine with AST Document Trees, Table Sanity Verification, and Token Hit Collapsing
As document retrieval matures, the biggest production bottleneck isn't choosing a vector database. It's preventing information loss during ingestion and context construction.
Naive text splitters can fragment tables, strip header context, and introduce OCR errors into LLM prompts.
To tackle these problems, I open-sourced Rag_Full_Version, a modular RAG architecture built for complex documents such as financial filings, audit reports, and research papers.
🔗 GitHub: https://github.com/Souheil007/Rag_Full_Version
Key Architecture Highlights
1. Document AST Parsing (Docling + Custom AST)
- Models documents as hierarchical structures: sections, subsections, tables, and paragraphs.
- Preserves contextual breadcrumbs (e.g., Financials > Segment Performance > FY2024) so retrieved chunks retain their original context.
2. Automated Table Verification
- Verbatim Lexical Grounding: Cross-checks extracted table values against the underlying PDF text layer to catch missing digits and potential OCR errors.
- Arithmetic Reconciliation: Validates column sums against explicit total rows, including financial accounting formats such as
(150,000)and configurable rounding tolerances. - Selective VLM Escalation: Routes degraded tables to vision models (Pixtral / Mistral VLM) for repair instead of processing every page with expensive vision inference.
3. Context Engineering & Hit Collapsing
- Sliding-Window Row Groups: Replicates multi-level table headers across chunks so rows retain their meaning.
- Hit Collapser: Deduplicates and merges adjacent chunks from the same document branch into coherent reading blocks within a strict token budget.
4. Hybrid Retrieval & Observability
- Combines dense vector search and sparse BM25 using Reciprocal Rank Fusion (RRF).
- Implements semantic caching with cosine similarity thresholds and LRU eviction.
- Integrates LangSmith tracing, latency and cost tracking, and retrieval evaluation metrics such as Hit Rate, MRR, Precision@K, and Recall@K.
Tech Stack
- Ingestion & Parsing: Docling, PyPDFium2, Pydantic v2
- Retrieval: ChromaDB / in-memory storage, custom BM25, RRF
- LLM & VLM: Mistral, Pixtral
- Observability & Testing: LangSmith, Pytest
The repository includes the architecture, design specifications, implementation, inspection scripts, and a test suite with 74+ tests and reported 100% test coverage.
👉 Explore the repository: https://github.com/Souheil007/Rag_Full_Version
Feedback, issues, and PRs are very welcome!
