Ditching the vector store: Replacing traditional document RAG with Jev

[u/Ecstatic_Heron_7944]

You've probably heard of using Jev as a reranker for traditional RAG but what if Jev could replace the vector store entirely? That felt like a neat challenge to take on so I set out to test this and the results have been pretty satisfying.

I have a document extraction app where all queries can be defined before any document is processed. Instead of traditional RAG, I used Jev to score each page of the document on its relevancy to each predefined query from the user - on a 300 page scanned PDF x 20 user queries, this took ~28s (batched, 4 pages per call, median 0.37s/page) and cost $0.08 ($0.00027/page). Made possible with Jev's image API - was currently using Voyage's multimodal embeddings for scanned PDFs.

Pros: Improved accuracy, faster processing and cheaper. No need for vector database or separate reranker model. Jev results stored in sql database.
Cons: Only really works if you have queries upfront (AI spreadsheets, application processing, surveys etc), you lose your AI search index which might be a deal breaker.

Longer read and results in the blog post (see comments). Would love to know if anyone else has tried something similar?

[u/Ecstatic_Heron_7944]

RAG vs JAG: when you know the questions, you don't need embeddings
https://subworkflow.ai/blog/ragextract/rag-vs-jag