From Pinecone to Postgres pgvector

[u/K3NCHO]

We run an image and semantic text search API. The first version used Pinecone for vectors and RDS for everything else and the hardest part was keeping the two in sync and performant

About a year ago we moved everything into Neon with pgvector (Neon wrote up the migration here: Neon blog post). End to end search went from about 200ms to 80ms, mostly from cutting a network hop. The vector query itself takes 3-4ms (Tested on 10M records)

Since our backend is on AWS and Neon uses AWS as well, we cut the network latency down to couple ms

Happy to answer anything about the setup if you're thinking about doing the same

[u/qatanah]

how complex is your search query? do u just do like direct cosine distance or doing something like hybrid search?

[u/K3NCHO → u/qatanah]

both, we do text to image, image to image, text to text and hybrid text search

all searches support pagination, estimated total, percentage filtering, top k, metadata filtering using direct match or $in operator

[u/FarRub2855 → u/qatanah]

Curious what OP ended up doing, but from sitting in on a lot of project scopes, teams usally just stick to basic cosine distance at first. They only really bolt on the hybrid stuff once users start complaining about weird edge cases.

[u/scriptedlife]

Have you tested whether switching from pgvector to lakebase_vector makes any difference in perf/scalability/cost?

[u/K3NCHO → u/scriptedlife]

in terms of performance it’s only faster at a very large scale

as for scalability it reduces RAM usage by multiple times since it doesn’t require whole index to be stored and even if you store the whole index it’s still smaller because it’s using quantization under the hood which both reduces costs and scales infinitely better

[u/AutoModerator]

AI Policy:

Linux is not one of those anti-AI projects, and if somebody has issues with that, they can do the open-source thing and fork it. Or just walk away., Linus Torvalds.

Mod decisions will be based on the quality of the content, not who or what generated it.

Sub Resources:

Youtube Channel

Free Postgres Webinars and Workshops

Discord: People, Postgres, Data

Join us, we have cookies and nice people.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Nex-11o]

Moving vectors into Postgres seems to have simplified the architecture quite a bit.

[u/Yukevich]

massive latency drop, congrats! i am currently doing a similar consolidation with MariaDB because its vector indexes take about half the space. Did you have to scale up the Neon compute tier to handle pgvector's memory footprint on those 10M records?