HNSW ef_search Parameter Explained: The Knob That Trades Recall for p99

The failure usually arrives as a quality complaint, not a latency one. Someone swaps the embedding model, reindexes over the weekend, p99 on the vector endpoint looks identical to last week, and ten days later product reports that RAG answers have gone vague. Nothing on the dashboard moved, because nothing on the dashboard measures whether the approximate neighbors were the actual neighbors. This is the hnsw ef_search parameter explained from the operator’s side: what the number does inside the graph, the one metric that tells you whether it is set correctly, and how to export that metric so the next regression pages someone instead of surfacing as a ticket.

What ef_search controls inside the graph

HNSW, from Malkov and Yashunin, searches a stack of proximity graphs. A query enters at the sparse top layer, greedily walks to the nearest node, drops a layer, repeats, and on the dense bottom layer runs a beam search. ef is the width of that beam: the number of candidates the search keeps alive while it expands neighbors. The hnswlib parameter notes define it as “the size of the dynamic list for the nearest neighbors (used during the search)”, note that higher ef is more accurate but slower, and add the constraint that ef cannot be set below k; anything from k up to the dataset size is legal.

Two operational consequences follow. It is a per-query cost, touching nothing on disk, so you can change it between requests without a rebuild. And it caps the candidate pool, which becomes a hard ceiling once you filter after the scan.

The same knob wears different names:

• hnswlib: index.set_ef(n).
• Faiss: index.hnsw.efSearch; the Faiss index guidelines call it the speed/accuracy tradeoff for HNSW.
• pgvector: hnsw.ef_search, default 40, settable per session or per transaction.
• Qdrant: hnsw_ef in search params; the indexing docs say it defaults to ef_construct, which is 100 by default.
• Weaviate: ef, default -1, meaning dynamic. The index reference derives it from the query limit via dynamicEfFactor (8), bounded by dynamicEfMin (100) and dynamicEfMax (500).
• Milvus: ef in search params, default equal to the limit, range [1, int_max].
• Elasticsearch: num_candidates, gathered per shard before the top k are merged; Elastic’s kNN docs call it the main search-time speed/accuracy control for HNSW.

The build-time cousins M and ef_construction decide which graph exists; ef_search decides how hard you search it. hnswlib’s own heuristic: if recall with ef set equal to ef_construction is still below 0.9, the problem is the build parameters, and no search-time value will fix it. See HNSW vs IVF tradeoffs for the memory side of raising M.

The metric that matters: recall@k against a brute-force golden set

recall@k = |ANN(q, k) ∩ exact(q, k)| / k, averaged over a held sample of real queries, where exact(q, k) comes from a flat scan of the same corpus.

It beats the obvious alternatives for three reasons. Latency is what you notice and recall is what fails silently, so latency alone cannot detect the failure above. Mean result distance is not comparable across embedding models or after a reindex. Recall@k needs no human labels: the “label” is the exact top-k, which you can compute with a numpy matmul or a Faiss IndexFlat. The deliverable is the pair (recall@k, p99) at each ef value, not either number alone.

Wiring it up

An hnswlib sweep that exports recall and latency to Prometheus, using the client_python Histogram:

import time
import numpy as np
import hnswlib
from prometheus_client import Gauge, Histogram, start_http_server

DIM, K = 768, 10
EF_VALUES = [16, 32, 64, 128, 256, 512]

recall_at_k = Gauge("ann_recall_at_k", "recall@k vs brute force", ["index", "ef"])
query_latency = Histogram(
    "ann_query_seconds", "ANN query latency", ["index", "ef"],
    buckets=[0.0005, 0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1],
)

def exact_topk(corpus, queries, k):
    scores = queries @ corpus.T          # unit-norm vectors: IP == cosine
    return np.argsort(-scores, axis=1)[:, :k]

def sweep(index, queries, golden):
    for ef in EF_VALUES:
        index.set_ef(max(ef, K))         # ef must be >= k
        hits = 0
        for i, q in enumerate(queries):
            t0 = time.perf_counter()
            labels, _ = index.knn_query(q, k=K)
            query_latency.labels("docs-v3", str(ef)).observe(time.perf_counter() - t0)
            hits += len(set(labels[0]) & set(golden[i]))
        recall_at_k.labels("docs-v3", str(ef)).set(hits / (len(queries) * K))

if __name__ == "__main__":
    start_http_server(9108)
    corpus = np.load("corpus.npy").astype(np.float32)
    queries = np.load("query_holdout.npy").astype(np.float32)
    index = hnswlib.Index(space="ip", dim=DIM)
    index.load_index("docs-v3.bin")
    sweep(index, queries, exact_topk(corpus, queries, K))

The pgvector equivalent scopes the setting to one transaction so a sweep never leaks into serving traffic:

BEGIN;
SET LOCAL hnsw.ef_search = 128;
SELECT id FROM items ORDER BY embedding <=> $1 LIMIT 10;
COMMIT;

Scrape it and alert on the gauge:

scrape_configs:
  - job_name: ann-recall
    scrape_interval: 60s
    static_configs:
      - targets: ["ann-eval:9108"]

groups:
  - name: ann
    rules:
      - alert: AnnRecallRegression
        expr: ann_recall_at_k{index="docs-v3", ef="128"} < 0.95
        for: 10m

What you’ll see

Good looks like a knee. Recall@k climbs steeply through the low ef values, then flattens; latency keeps climbing roughly in step with ef, because every extra candidate is another full-vector distance computation. Pick the smallest ef past the knee whose p99 fits the budget, pin it in config, and keep the sweep job running against every reindex.

Three bad shapes:

1. The curve plateaus under target. Raising ef past the plateau buys latency and nothing else; per the hnswlib guidance, that is an M or ef_construction problem and needs a rebuild.
2. Recall is fine but p99 spikes under concurrent load at high ef. HNSW is memory-bandwidth bound; Faiss puts the footprint at (d × 4 + M × 2 × 4) bytes per vector, and once the graph plus vectors stop fitting in RAM the latency curve stops being smooth.
3. Filtered queries return fewer than k rows. pgvector applies the WHERE clause after the index scan, so ef_search caps how many rows exist to filter. Version 0.8.0 added hnsw.iterative_scan (strict_order or relaxed_order), bounded by hnsw.max_scan_tuples (20,000 default), for exactly this case. Weaviate’s dynamic ef sidesteps part of it by scaling with the limit.

Most of the causes of low vector search recall show up on this one chart before they show up anywhere else.

Caveats

• ef below k. hnswlib refuses it; Milvus silently defaults ef to the limit. A client that raises limit from 10 to 100 without touching ef gets engine-dependent behavior. Clamp explicitly, as the sweep above does.
• Stale golden set. Exact top-k is a snapshot of one corpus under one model. After a reindex or embedding swap, recompute it, or the alert fires on a comparison that no longer means anything.
• Query leakage. Tune ef on one query sample and report recall on a held-out one. Tuning and reporting on the same queries overfits ef to that sample, the ANN analog of label leakage.
• Sampling cost. Brute force is O(N × d) per query. Run a few thousand held-out queries on a schedule, never on the serving path.
• Cardinality. ef as a label is fine for a bounded sweep list. Never attach it, or a query id, to the serving histogram.
• Defaults are not benchmarks. pgvector’s 40, Qdrant’s 100, Weaviate’s 100 to 500 window and Milvus’s ef = limit are vendor starting points tuned to each engine’s default M and ef_construction. None is an independent benchmark on your data, and the HNSW paper’s own comparisons are author benchmarks, not third-party ones.

Put the recall gauge on the same board as your model drift panels; the monitoring patterns at SentryML apply unchanged, because a recall regression after reindex is drift in everything but name.

Related across the network

• LangChain Building Blocks: Chains, Tools and Agent Control — langchainguide.com
• LangChain RAG Pipeline: Setup to First Answer — langchainguide.com
• LangChain vs LangGraph vs LlamaIndex Compared — langchainguide.com
• Qdrant vs Milvus vs Pinecone: Vector DB Comparison — ragstackguide.com
• RAG Chunking Strategy: Picking Chunk Size and Overlap — ragstackguide.com