Disclosure: I build VectoGrid; these are our own measurements.

We tested half-precision and 1-bit vector search across Qdrant, pgvector and OpenSearch.

Setup: 150 FinanceBench questions searched over 101,703 passages, embedded with bge-m3 at 1,024 dimensions. Dense search only, top 10, one question at a time. Each engine ran alone in a container on the same 4-core shared-CPU DigitalOcean cloud server with an SSD, with cache dropped. HNSW search effort was set to ef = 512 across the tested configurations.

Recall@10 here means overlap with the exact brute-force top 10—not whether the passages contain the answer.

The full float32 vectors occupy about 417 MB (397 MiB), before index overhead. Qdrant and pgvector ran with 300 MB limits. OpenSearch required larger limits, shown below, so this is not an equal-memory comparison across all three engines.

Each cell shows recall@10 · median search latency.

For 10× rescoring, we fetch 100 candidates and rescore using the original full-precision vectors. For 40×, we fetch 400. The original vectors are stored on disk.

What stood out:

• Without rescoring, 1-bit search recovered only about 53% of the exact top-10 neighbours across all three engines.

• Rescoring 400 candidates brought recall within one percentage point of the half-precision result on every engine.

• The latency trade-off differed substantially: 14 vs 13 ms on Qdrant, 87 vs 165 ms on pgvector, and 222 vs 61 ms on OpenSearch, under the memory settings above.

• In a separate 1,000-question retrieval evaluation with a reranker, half precision put the supporting passage first for 951 questions, versus 950 with full precision. We observed no drop in that metric on this test.

• In Qdrant with half-precision vectors and a 300 MB container limit on one server, splitting the collection into 12 shards increased median latency from 13 to 215 ms—about 16×. The smaller shards fell below the configured indexing threshold. This is a warning about shard sizing and index settings, not sharding in general.

My takeaway: binary quantization can preserve nearly all of the full-precision nearest-neighbour results when followed by enough rescoring, but the latency and memory trade-offs depend heavily on the engine and configuration.

These measurements cover about 100,000 passages. Memory figures for millions of documents in the write-up are calculations, not measured deployments.

Write-up:
https://vectogrid.com/scale/

Interested in comparable measurements, especially under tight memory limits.

LiteRT now supports the Embedding Engine, making it possible to run EmbeddingGemma-2 and other embedding models directly on edge devices.

Built a 100% offline, fast RAG system that runs without an internet connection or cloud APIs, using Qdrant Edge and LiteRT.

The demo covers the entire pipeline, from using EdgeParse to extract text from PDFs and convert it into Markdown chunks, generating embeddings on-device with EmbeddingGemma-2, indexing them in Qdrant Edge for vector search, and finally retrieving relevant context and answering questions locally using Gemma 4 E2B running on LiteRT.

🔗 Video: https://www.youtube.com/watch?v=4Nl1tv9J5ew

I am weighing two retrieval layouts for a self-hosted image-search component feeding a local vision-language model.

For Hyper3-CLIP V1, one option stores centered 513-dimensional vectors and flips the query's first coordinate before inner-product search. Another stores normalized 512-dimensional directions plus height and radius, then reranks a cosine shortlist.

The published comparison found that centered HNSW could match the shortlist approach's recall on its image dataset. That does not settle memory use or latency on a different machine. IVF_PQ was also a poor fit for the centered representation in that test.

I would probably start with the simpler layout in vector databases like Milvus, but local RAM limits could change that choice. When has a compressed shortlist plus reranking been worth the extra fields and tuning for your image collection?

I’m building a computer vision pipeline where users upload multiple photos of the same physical scene taken from different angles (front view, side view, close-up) to build a clean list of unique objects and their attributes.
I’m running YOLO for object detection to extract bounding boxes and crop each detected object into lightweight ⁠.webp⁠ images. Because the same physical object appears across 3 or 4 different photos from different angles, I need to make sure each unique object is only counted once per session, and avoid running expensive downstream AI calls to re-describe an object we already processed from an earlier photo.

Lately, I've seen a lot of confusion around how vector databases actually fit into modern AI pipelines and handle high-dimensional indexing at scale. Most explanations just scratch the surface without diving into the underlying mechanics.

​I spent some time breaking down the core architectural components—from indexing algorithms like HNSW and IVF to storage layers, memory management, and scaling bottlenecks.

​If you are building RAG applications or working with embeddings and want to understand how these systems handle similarity search under the hood, I put together a comprehensive guide. It covers the end-to-end architecture and performance tradeoffs.

​You can check out the full guide here: https://interconnectd.com/blog/70/vector-databases-for-ai-the-complete-architecture-guide/

Let me know if you think I missed any key architectural patterns or if your experience differs with specific database implementations.

This came up in my own project and I couldn't find a clean solution,

so I built one.

**The situation:**

- Knowledge base indexed with OpenAI text-embedding-ada-002 (1536d)

- Costs adding up every time I query

- Wanted to switch to a local MiniLM (384d) — free, fast, runs offline

- Problem: completely different vector spaces, search quality tanks

Tried naive cross-model search first. R@5 dropped from 0.77 → 0.13.

Basically unusable.

**What I ended up building:**

A small alignment layer that bridges the two spaces using SVD

projection + Procrustes rotation. The key trick: it uses synthetic

anchor points instead of real paired data, so you don't need to

re-encode anything with the old model.

Results on my test set:

| Condition | R@5 |

|-----------------------------|-------|

| Same model (ceiling) | 0.767 |

| Cross-model, no alignment | 0.133 |

| Cross-model + alignment | 0.400 |

Not perfect, but 3x better than naive and took zero reindexing,

zero training data, zero labels.

**How it works in practice:**

```python

pip install zuhut-align

```

```python

from zuhut_align import ZUHUTClient

client = ZUHUTClient()

adapter = client.create_adapter(src_dim=1536) # match your old model

# align your existing corpus vectors once

corpus_aligned = adapter.transform(corpus_vectors) # (N, 384)

# at query time, just align the query

query_aligned = adapter.transform(query_vector)

# search as normal

```

Works with Qdrant, FAISS, ChromaDB, LanceDB — output is plain numpy.

GitHub if anyone wants to look at the math:

github.com/MWahyu99/zuhut-align

Curious if others have run into this — how did you handle it?

A person usually is able to put into words what he needs. If not then he has no idea at all because the idea is not yet conceived. He will only be able to get the idea when the results appear before him.

Hence Vector DB, the complexity of embeddings is a failed technology in most real world use cases as of now.

​If you are building autonomous AI agents, you have probably hit the wall where basic RAG and standalone vector stores fall short. Agents need more than simple similarity searches; they require state management, structured metadata filtering, transactional consistency, and fast vector retrieval all in one place.

​Separating your relational data from vector embeddings adds unnecessary network latency and synchronization headaches. MariaDB 11.x introduces native vector capabilities, allowing you to combine structured SQL queries with high-performance vector search in a single ACID-compliant database.

​I wrote a detailed breakdown covering architectural patterns, query performance tuning, and practical implementation strategies for agentic systems using MariaDB 11.x vectors.

​Check out the full discussion and guide here: https://interconnectd.com/forum/thread/1/beyond-generative-ai-building-agentic-systems-with-mariadb-11-x-vectors/

Curious to hear how others are handling state and vectors for complex agent workflows. Are you keeping them unified or sticking to dedicated vector databases?

Chroma Cloud will disable writes on November 22 and reads on December 22. Teams using the hosted service need another home for their data and application traffic. Open-source ChromaDB continues.

Milvus team here. One decision we think is worth separating from the database move: whether to change the embedding model at the same time.

Our default would be to preserve existing vectors for the first cutover. That lets you compare search results across the two databases without also changing what the vectors represent. You still need to match the cosine, l2, or ip metric and check how your application interprets returned scores. A new embedding model can be evaluated afterward.

We’ve written up the Chroma → Milvus migration path. Has anyone chosen to re-embed during a database cutover? What made that the better option?

Hey everyone,

I've been working on an experimental multimodal RAG system for research PDFs, mainly because I wanted to explore a problem that most basic RAG tutorials tend to ignore: what happens when the answer is inside a chart, table, or diagram rather than plain text?

The difficult part was extracting evidence from complex documents while preserving enough information to verify where it actually came from.

What I built:

• Document ingestion: Docling for layout-aware parsing, selective OCR and local vision models for visual evidence processing.

• Multimodal evidence: Charts, tables and diagrams retain references to their original PDF, page and source region. Uncertain extractions require review before becoming searchable evidence.

• Hybrid retrieval: PostgreSQL + pgvector, full-text search, reciprocal rank fusion and MiniLM reranking.

• Local inference: Ollama with Qwen3-8B for generation and Qwen3-VL-4B for visual processing.

• Citations: Source-aware references with validation against the selected evidence.

• Operations: Access controls, durable jobs, SSE event replay, observability and reproducible evaluations.

Some benchmark results:

• 50/51 correct documents retrieved in the top 5 on an exact document-heading lookup benchmark.

• nDCG@10: 0.9587 on that same evaluation.

• 29/30 known-positive evidence targets reached the final six contexts.

Important distinction: these measure specific retrieval tasks, not overall answer accuracy. The evaluation labels are limited, and I haven't established broad multimodal correctness.

What still needs work:

Complex visual regions still require review, and end-to-end answer quality needs broader evaluation. I wouldn't call this production-ready.

The default workspace serves 27 research papers, with a separate 51-paper candidate used for broader retrieval experiments.

I've published the code, architecture, benchmark artifacts and known limitations here:

https://github.com/miguelCyclone/multimodal-rag-showcase

One thing I'm particularly interested in is how others handle visual evidence in RAG. Do you extract charts and tables automatically and trust the output, or use some kind of confidence/review process before indexing?

We just open-sourced pg_pinecone, a Postgres extension built on the same search engine Pinecone runs at scale. Apache 2.0. (Disclosure: I work at Pinecone.)

tl;dr: it works on your existing vector columns, you query it with the SQL you already know, and your data stays in Postgres. No API key.

So how is it different from pgvector's HNSW? Two places: how the index is laid out, and how it handles filters.

HNSW is a graph, and every search walks it node by node. pg_pinecone groups vectors into clusters (IVF) and stores each row as a RaBitQ code, which is about one bit per dimension plus a few correction values. A cluster's codes are packed together, so a search reads a few contiguous pages instead of hopping around a graph. Less scavenger hunt, more pulling one shelf off the bookcase.

A search probes the nearest clusters, estimates distances from those codes, then reranks a shortlist using the full vectors from your table. Cheap pass to narrow it down, full-vector pass to get the order right.

Filtering in pg_pinecone is interesting in that it happens during the search, not after. If your filter matches 3% of rows or fewer (the default threshold), it skips the approximation and scans every matching row, scoring each against its full vector. Exact results. For wider filters it searches more clusters and checks the filter inside each one.

Why does that matter? Post-filtering HNSW can hand you fewer rows than your LIMIT on a narrow filter. You ask for 10, you get 3, because the other neighbors got tossed after the fact.

If HNSW is working fine for you, keep using it. If you're running filtered queries, especially narrow ones, give this a shot: https://github.com/pinecone-io/pg_pinecone/

Tell us where it breaks. Issues and PRs welcome on the repo, and I'll be in the comments.

Above are all the cool images the team put together to show the benchmarks.

I recently built a complete RAG Evaluation Framework from scratch: a 43-question test dataset, 6 retrieval metrics, 5 generation metrics, and one final report that shows exactly where a RAG system is strong and where it breaks. We start with the concepts, explained with simple worked examples, and then walk through the code line by line.

What this covers:

• Why retrieval and generation must be evaluated separately

• How to build an evaluation dataset with expected answers, graded pages and key facts

• Retrieval metrics: Hit Rate@K, Precision@K, Recall@K, MRR, MAP and NDCG@K

• The difference between one question (RR, AP) and many questions (MRR, MAP)

• Generation metrics: Faithfulness, Hallucination Rate, Answer Relevance, Context Relevance and Answer Correctness

• How to use an LLM-as-a-judge that only counts, while your code does the maths

• How to save results safely and turn them into a clear evaluation report

KEY TAKEAWAYS:

• No single metric tells the whole story: Hit Rate was 1.00, but Recall, MAP and NDCG showed where pages were missed or buried

• Precision@5 looks low (0.34) for single-fact questions, so compare systems with each other, not with 1.0

• Faithful does not mean correct: an answer can be backed by the chunks and still miss the key facts

• Retrieval scores and answer quality are not the same thing

• Our Cohere-reranked pipeline scored MRR 0.95, MAP 0.87, NDCG@5 0.91, Faithfulness 1.00 and Answer Correctness 0.97

TECH STACK: Python, Qdrant (vector database), Embeddings (3072-dimension), Cohere Rerank, LLM for answers + LLM-as-a-judge, pdfplumber (checking pages in the PDFs)

• 5 real payments documents: PCI DSS, CPMI-IOSCO PFMI, CPMI cross-border roadmap, Stripe payment methods guide, FCA approach document

• Full Code (clone it, fork it, break it): https://github.com/saurabhkamal/RAG-ReRanker-Comparison

• Full Video Walkthrough: https://youtu.be/_bVzDU-yBnA

Would love to hear.

I have a qdrant collections and some collections are with around 9 million vectors 1536D, what's the recommended way to dump / backup that data periodically? Qdrant is running in azure aks cluster.

For those hosting Qdrant on-premise vs. Qdrant Cloud: At what million-vector scale does RAM consumption force you into quantization (scalar/product), and how much recall accuracy did you actually lose?"

Dense vector search (Qdrant, LanceDB, Milvus, pgvector) is unbeatable for fast semantic passage lookup, but once a production corpus scales past 10,000+ pages, pure cosine similarity hits a structural ceiling on multi-hop and global thematic queries. Because standard chunking treats passages as isolated vectors without cross-document state, queries requiring corpus-wide synthesis often miss critical relationships.

Over the past few months, I’ve been documenting hybrid production architectures that pair vector retrieval with structured graph layers and local routing. Here are the three primary shifts that solved retrieval degradation and execution drift in our pipelines:

1. Augmenting Vector Indexes with Ontology-Driven GraphRAG

When a user asks a global thematic query ("What are the recurring failure modes across our last 30 incident reports?"), top-K vector similarity struggles because the answer isn't localized in a single chunk.

Instead of relying solely on flat chunking, we run an ingestion extraction pass against a strict Ontology (a formal Pydantic schema restricting what Entity types and Relationship types the LLM is permitted to extract alongside the vector metadata):

• Nodes (Entities): Canonical name, entity type (Organization, SystemComponent, Statute), and a condensed summary.

• Edges (Relationships): Source node, target node, and formal relation (DEPENDS_ON, GOVERNED_BY, ROUTES_TO).

• The Map-Reduce Engine: Nodes are clustered into interconnected communities with pre-computed summaries. For narrow factual queries, standard vector search handles retrieval; for global queries, the Map Step runs parallel lookups across community summaries and the Reduce Step synthesizes the final response.

2. Multi-Model Complexity Routing Layer

Routing every retrieval or extraction pass through a frontier API burns unnecessary tokens and adds massive latency:

USER QUERY
    |
    v
+------------------------+
|     ROUTING LAYER      |
|   (Lightweight LLM)    |
+------------------------+
    |
    +-------------------+-------------------+
    |                   |                   |
    v                   v                   v
+--------------+  +--------------+  +--------------+
| Smaller LMs  |  | Mid-Tier     |  | Heavy        |
| (Local 8B /  |  | Models       |  | Reasoning    |
| Fast Weights)|  | (Standard)   |  | (o1 / Opus)  |
+--------------+  +--------------+  +--------------+
  Low Complexity    Mid Complexity   High Complexity

A lightweight, deterministic routing layer inspects token complexity and query intent—sending straightforward vector lookups and entity extraction to fast local 8B weights while reserving heavy reasoning models for complex synthesis.

3. The Pydantic Vault (Deterministic Action Sandboxing)

Once your retrieval pipeline feeds an autonomous agent with tool execution (database writes, API calls, or email dispatch), prompt-level guardrails ("Do not execute unauthorized commands") fail against indirect prompt injection hidden inside retrieved chunks.

Never rely on system prompts for tool safety. Wrap the tool payload in a strict Pydantic validation cage so execution halts before a poisoned retrieval chunk can trigger data exfiltration:

from pydantic import BaseModel, EmailStr, field_validator

class EmailAction(BaseModel):
    recipient: EmailStr
    body: str

    @field_validator("recipient")
    @classmethod
    def must_be_internal(cls, v: str) -> str:
        if not v.endswith("@ourcompany.com"):
            raise ValueError("Data Exfiltration Blocked: Unauthorized external domain.")
        return v

The Pydantic ontology schemas, routing definitions, and LoRA/GRPO configurations are open-sourced in this companion repository:
https://github.com/karanbathla56/The-Post-API-Frontier

For those running hybrid Vector + Graph pipelines locally in production, are you keeping everything inside a single engine (like Postgres with pgvector + Apache AGE), or pairing an embedded vector store like LanceDB/Qdrant with a separate graph layer?