Vector Databases

Also Known As

TL;DR

Explanation

Traditional databases cannot efficiently query 'find the 10 most similar vectors to this query vector' across millions of rows. Vector databases use specialised index structures (HNSW, IVF) for approximate nearest-neighbour (ANN) search. Options: Pinecone (managed), Weaviate (self-hosted or managed), Qdrant (Rust, self-hosted), pgvector (PostgreSQL extension — good starting point). For PHP applications, pgvector requires no new infrastructure and supports hybrid search (vector + SQL filters).

Common Misconception

Why It Matters

Common Mistakes

• Not creating an index on the vector column — without HNSW or IVF index, queries are O(n) exact search.
• Storing vectors as JSON strings — use the native vector type (pgvector's vector type) for efficient indexing.
• Not filtering by metadata before vector search — combining SQL WHERE clauses with vector search (hybrid search) dramatically reduces the search space.
• Choosing a hosted vector DB before trying pgvector — pgvector handles millions of vectors adequately and eliminates an extra service.

Code Examples

// Exact search over all vectors — O(n), unusable at scale:
SELECT id, content,
       embedding <=> $1 AS distance
FROM documents
ORDER BY distance
LIMIT 10;
-- No index: scans all rows, 1M docs = seconds per query

-- pgvector with HNSW index — O(log n) approximate search:
CREATE EXTENSION IF NOT EXISTS vector;
ALTER TABLE documents ADD COLUMN embedding vector(1536);
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);

-- Hybrid search: filter then vector search:
SELECT id, content, embedding <=> $1 AS distance
FROM documents
WHERE category = 'security'   -- SQL filter reduces search space
ORDER BY distance
LIMIT 10;

References

• ↗ https://github.com/pgvector/pgvector
• ↗ https://qdrant.tech/documentation/

Tags

Related Terms