How does a vector database find the right passage?
Every document chunk is stored as an embedding, a vector, alongside the original text and any metadata like source or date. A query arrives, gets embedded the same way, and the database runs an approximate nearest-neighbor search to return the closest vectors in milliseconds, even across millions of chunks. That approximate search is the entire mechanism a RAG system leans on to find grounding text before the model ever sees the question.
What should actually decide which vector database to use?
Scale, filtering needs, and how much operational surface you want to own. A managed vector database trades control for less infrastructure to run; pgvector inside an existing Postgres instance trades some scale ceiling for one fewer system in the stack. What rarely matters as much as vendors imply is raw benchmark speed: chunking strategy and embedding quality usually decide retrieval accuracy before the database engine does. Metadata filtering, restricting a search to one customer’s documents or one date range, is often the feature that decides fit more than search speed, since a fast search over the wrong subset of documents is still the wrong answer, feeding stale context into the LLM’s prompt.