25 Apr 2026 AI Engineering 6 min read

Vector store consolidation: the deployment model decides

By 2026 the vector store decision is no longer an ANN benchmark. pgvector 0.8.2 wins where Postgres is already the data platform, Qdrant 1.17 where filtered search and memory efficiency at scale matter, Azure AI Search where a managed Azure estate needs hybrid retrieval and semantic ranking. With memory math, residency and the case for no vector database at all.

The choice is settled before the benchmark

Retrieval systems accumulate vector stores. A pilot on one engine, a production index on another, a third instance that a different team provisioned. By April 2026 there is no technical justification left for this. The mature engines all implement HNSW with comparable recall and latency; the published gaps closed years ago. What did not converge is how the stores are operated, billed, patched and audited. Consolidation is therefore an operations decision, not a retrieval-quality decision.

The binding constraints are where the vectors may physically reside, who patches the process, and which team carries the pager. Three options cover nearly every estate we see. The table states the mapping we apply; the sections that follow state what each option does not do.

Deployment constraint	Option	Hybrid and reranking	Data residency
Postgres already in production	pgvector 0.8.2	RRF hand-written in SQL; BM25 via extension	Wherever the database already runs
Self-hosted Kubernetes	Qdrant 1.17	RRF and DBSF built in; late interaction rerank	Full control including on-premises
Managed but sovereign	Qdrant Hybrid Cloud	Same as self-hosted Qdrant	Data plane in your cluster; telemetry leaves
Azure estate with managed identity	Azure AI Search 2026-04-01	Hybrid RRF plus managed semantic ranker	Region-pinned Azure resource
Under one hundred thousand chunks	No dedicated store	Exact scan plus Postgres full-text search	Follows the primary database

A document arrives — scan, mail or upload. 1/4

pgvector when Postgres is the data platform

pgvector 0.8.2 shipped on 25 February 2026. It provides HNSW and IVFFlat indexes, the halfvec type at two bytes per dimension and indexable up to 4000 dimensions, bit vectors with binary_quantize, and sparsevec. The more consequential change came with 0.8.0: iterative index scans. Before it, a query combining an ANN index with a restrictive WHERE clause silently returned fewer rows than requested. Now the scan continues until the limit is satisfied. That single fix removed most production surprises.

What pgvector does not provide: hybrid search and reranking. Postgres full-text search yields tsvector and ts_rank, not BM25; BM25 requires a third-party extension. Reciprocal rank fusion is a common table expression you write and maintain yourself. Index builds are bound by maintenance_work_mem and become slow once they spill to disk. In return the vectors sit in the same transaction, the same backup and the same role model as the business data. For most teams that is the decisive argument.

Qdrant for filtering and memory efficiency

Qdrant 1.17 was released on 20 February 2026. Its argument is filtered search. Version 1.16 of 19 November 2025 added ACORN, which traverses second-hop neighbours when direct neighbours fail the filter. On a five-million-vector set Qdrant measured 97.20 percent accuracy at 13.86 milliseconds with ACORN, against 53.34 percent at 1.25 milliseconds without it. The fast configuration is the one that returns half the correct results. Low-selectivity filters are where naive HNSW quietly degrades.

The same release introduced inline storage, which places quantized vectors inside the HNSW nodes: 211 queries per second at 430 MiB on one million vectors, against 20 without it, close to the 334 that required 530 MiB of resident RAM. The Query API performs RRF and DBSF fusion natively, supports sparse vectors with built-in IDF, and offers late interaction over multivectors as a rerank stage. Qdrant does not choose your embedding model. The vectors still arrive from your own pipeline.

Azure AI Search for managed Azure estates

The stable REST API 2026-04-01 made knowledge bases and knowledge sources generally available, alongside index aliases, markdown parsing modes and user-assigned managed identities for indexer connections. Hybrid retrieval is native: BM25 and vector queries execute in parallel and merge by reciprocal rank fusion. The semantic ranker then rescores the merged list with multilingual models adapted from Bing and returns captions and optional answers. None of this requires infrastructure you operate.

The scoping matters. The semantic ranker reranks the top 50 results only; it never reruns the query over the corpus, so a document missing from the first stage stays missing. Each candidate is summarized to at most 2048 tokens, with 128 reserved for title and 128 for keywords, which makes field order in the semantic configuration a relevance decision. Billing is usage-based with a free monthly allowance, separate from agentic retrieval since April 2026, and the feature is available in selected regions only.

Quantization and the memory math

Do the arithmetic before the benchmark. Ten million chunks at 1536 dimensions in float32 are 61.4 GB of raw vectors. Qdrant's sizing rule adds roughly fifty percent for HNSW links, so about 92 GB resident. int8 scalar quantization reduces the vector payload to 15.4 GB. One-bit binary quantization reduces it to 1.92 GB. Qdrant added 1.5-bit and 2-bit variants in 1.15 at 24x and 16x compression for the middle ground. These are the numbers that decide the instance size.

Compression is lossy and the vendors say so. Microsoft documents int8 at four times reduction and binary at up to 28 times, not the theoretical 32, and recovers quality through oversampling and rescoring. For scalar quantization rescoring requires the full-precision vectors to be retained, which cancels part of the storage saving; for binary quantization the originals can be discarded and rescoring uses the binary dot product instead. Choose compression and rescoring together or the memory figure is fiction.

Why synthetic QPS numbers mislead

Public ANN benchmarks report recall against queries per second on static unfiltered datasets, single node, no payload, no concurrent writes, no tenants. Production retrieval inverts every one of those conditions. Each query carries a tenant or permission filter, the corpus is written while it is read, and one index serves many customers. The ACORN figures above are the clearest illustration: the configuration that is ten times faster is the configuration that is wrong.

Throughput is also rarely the binding constraint. In a retrieval-augmented request the generation step dominates end-to-end latency, and a ten-millisecond difference in the vector query disappears inside it. Missing recall under a restrictive filter does not disappear; it surfaces directly as a wrong answer. Measure on your own corpus, with your own filters, at your own write rate. Everything else is marketing arithmetic.

When no vector database is needed

State this plainly, because it saves the most money. Below roughly one hundred thousand chunks an exact scan over a pgvector column usually stays inside the latency budget, needs no index, and therefore loses no recall. If the retrieval is really a lookup by identifier, tenant or date, a WHERE clause on an ordinary B-tree beats any ANN structure. If the corpus is keyword-dense, BM25 alone often outperforms embeddings and is far easier to explain to auditors.

A second datastore is not a library. It adds a consistency boundary, a backup regime, an access-control model, a patch cadence and a migration path for the day the embedding model changes. We introduce a dedicated engine when filtered recall, memory efficiency or tenant isolation demonstrably fail in Postgres. Not before, and not because a benchmark chart looked convincing.

What we expect for the rest of 2026

The differentiator has already moved from ANN quality to operations: multitenancy, memory tiering, filtered recall at low selectivity, and retrieval APIs that agents call directly. The Azure release of April 2026 shows the pattern. The index stops being a component you assemble and becomes a managed retrieval service you call. Independent engines keep the ground where residency, cost per gigabyte and on-premises operation decide. That ground is not shrinking in Europe.

The real migration cost is the embedding model, not the store. Every model change re-embeds the corpus and invalidates tuned index parameters. We expect the next round of competition to be about swapping vector fields in place and re-embedding incrementally. Our working assumption for the rest of 2026: one Postgres-based store plus at most one dedicated engine. Three stores in one estate is an audit finding.