pgvector for RAG: HNSW, IVFFlat, Filtering and Tenant Isolation

pgvector is attractive because it lets a team keep embeddings close to the documents, metadata, transactions, and permissions that already live in PostgreSQL. That does not make vector search automatic. The hard questions are still there: which records may be searched, how much recall is enough, and what happens when a filter removes most of the nearest neighbours?

Start with exact search and a real query

Before choosing an approximate index, measure exact nearest-neighbour search on representative data. It gives you a useful recall baseline. Include the filters your application will actually use: tenant, collection, document status, language, and effective date. A benchmark that searches every document without those predicates can give a very misleading result.

HNSW and IVFFlat solve different problems

HNSW is often a strong default when you care about recall and want good query performance without a separate training step. IVFFlat can be a sensible choice when build time, memory, or a large regularly refreshed collection makes its trade-offs attractive. Neither index removes the need to measure. Tune the search breadth and compare results with the exact baseline.

Filters are part of retrieval quality

A vector that is close in meaning but belongs to the wrong tenant is not a near miss; it is a security failure. Apply tenant and permission constraints in the database query, not in a prompt after retrieval. With approximate indexes, selective filters can also reduce the number of acceptable results found. The official pgvector documentation covers iterative scans and related tuning options for this problem.

Keep the index and the boundary together

Use metadata columns that can be indexed, keep the embedding model and distance metric recorded, and re-run evaluation when chunking or filters change. Test empty tenants, deleted documents, permission changes, and a query designed to retrieve another organisation’s data. pgvector is a useful building block, but the RAG system is only as trustworthy as the SQL around it.