If you've tried to build search that understands meaning rather than just matching keywords, you've likely encountered vector databases. Here's what I learned integrating one into a production search pipeline.
The Problem with Traditional Search
Keyword search (TF-IDF, BM25) works well when users know exactly what to search for. But it fails spectacularly for semantic queries. Searching "how to handle errors gracefully in API responses" won't match a document titled "REST Exception Handling Patterns" — even though they're about the same thing.
What Are Vector Embeddings?
Embedding models (like OpenAI's text-embedding-ada-002 or open-source alternatives like Sentence-BERT) convert text into high-dimensional numerical vectors. Similar meanings cluster together in this vector space. The key insight: you can measure semantic similarity by computing the distance between vectors.
Choosing a Vector Database
I evaluated Pinecone, Weaviate, Milvus, and pgvector. For our use case (under 1M documents, PostgreSQL already in the stack), pgvector won on operational simplicity. For larger scales, dedicated solutions like Pinecone offer better performance.
The Architecture
Our pipeline: documents are chunked, embedded via API, and stored with their vectors. At query time, the search query is embedded using the same model, and we perform an approximate nearest neighbor (ANN) search. We combine this with metadata filtering for hybrid search.
Lessons Learned
Embedding quality matters more than the database choice. Chunk size dramatically affects retrieval quality — too large and you lose precision, too small and you lose context. And always build an evaluation set before optimizing.