Hybrid Search: BM25, Vector Retrieval, and Score Fusion

The failure that sends most teams looking for hybrid search is specific. A support assistant is asked about error 0x80070057 on the nightly backup job, the embedding index returns five plausible chunks about backup failures and misses the one containing that literal error code. The reverse failure is just as common: a keyword index returns nothing for “the backup dies overnight” because no document uses the word “dies”. Here is hybrid search BM25 and vectors explained from the operator’s side: what each retriever is good at, why their scores cannot simply be added, and what the fusion knobs in Elasticsearch, Weaviate, Qdrant and Pinecone actually compute.

Two retrievers with opposite failure modes

BM25 is a bag-of-words scoring function over an inverted index. Each query term contributes a weight that rises with term frequency, saturates (the tenth occurrence adds less than the second), is discounted for document length, and is scaled by inverse document frequency so rare tokens count more. k1 controls saturation and b controls length normalization; Robertson and Zaragoza report that 0.5 < b < 0.8 and 1.2 < k1 < 2 work well across many collections, and Elasticsearch ships with k1 = 1.2 and b = 0.75. BM25 matches only tokens literally present: weak on paraphrase, strong on part numbers, error codes, function names and proper nouns, all rare tokens with high IDF.

Dense retrieval encodes the query and every chunk into a vector and finds nearest neighbours through an approximate index such as HNSW. It tolerates paraphrase and synonyms because the encoder learned them. It is weak precisely where BM25 is strong: an identifier the encoder never saw during training gets compressed into a vector that carries little of its identity. The BEIR benchmark, which evaluates retrievers zero-shot across 18 datasets, found that BM25 “is a robust baseline” while dense retrievers often underperform it on domains they were not trained on. For background on the dense side, see vector search fundamentals.

Hybrid search runs both legs and merges the two ranked lists; the whole argument for it is that the failure modes are uncorrelated.

Why the scores cannot be added raw

A BM25 score is an unbounded positive number that grows with term frequency and the number of query terms. A cosine similarity lives in [-1, 1]. Pinecone’s documentation states the consequence plainly: without normalization, sparse scores dominate because they are unbounded while dense scores are not. Every hybrid implementation is therefore really a choice of fusion function, and there are two families.

Reciprocal rank fusion

RRF throws the scores away and uses only rank positions. For each document, sum 1 / (k + rank) across every list it appears in, then sort by the sum. Cormack, Clarke and Buttcher introduced it at SIGIR 2009 with k = 60, fixed during a pilot study, and reported that it beat Condorcet Fuse and CombMNZ on TREC runs. Because it never looks at a raw score, it cannot be wrecked by a scale mismatch between legs.

The constant is not standardized. Elasticsearch’s rrf retriever defaults rank_constant to 60 and fuses only the top rank_window_size results from each leg (default: the request size). Weaviate’s rankedFusion uses 1 / (rank + 60). Qdrant’s RrfQuery uses zero-based ranks with k defaulting to 2, which weights the top few positions far more steeply than k = 60 does. “We use RRF” means different things on different engines.

Normalized score combination

The second family normalizes each leg’s scores onto a common range, then takes a weighted sum: alpha * dense + (1 - alpha) * sparse. In Weaviate, alpha = 0 is pure keyword and alpha = 1 is pure vector; relativeScoreFusion, which min-max normalizes each leg before summing, has been the default since v1.24. Pinecone applies the same weighting by scaling the dense and sparse query vectors before a single dot-product query. Qdrant’s distribution-based score fusion (DBSF) normalizes on each leg’s mean and standard deviation, with 3-sigma extremes as endpoints, instead of min and max.

Bruch, Gai and Ingber analysed both and found that the convex combination outperformed RRF in-domain and out-of-domain, that RRF was sensitive to its parameters, and that tuning alpha needed very little labelled data. Weaviate’s own FiQA runs showed relative score fusion recalling about 6 percent more than ranked fusion; that is a vendor benchmark on one dataset. The Qdrant docs give the practical rule: score fusion when you trust the legs’ scores to carry magnitude and have an eval set to tune against, RRF when you do not.

The metric that matters: recall@k of the fused list

The fused score is meaningless on its own. The number to track is recall@k on the merged list, measured on a golden set of real queries with labelled relevant chunks, and compared against each leg run alone at the same k. If the fused list does not beat the best single leg, the fusion is misconfigured, whatever the score column says.

Anthropic’s contextual retrieval write-up gives the shape of the gain; it is a vendor benchmark on their own eval set. Defining failure rate as 1 minus recall@20, contextual embeddings alone reduced it from 5.7 to 3.7 percent, adding a contextual BM25 leg took it to 2.9 percent, and a reranker on top took it to 1.9 percent. The BM25 leg took a further 0.8 points off after the embedding change had taken 2.0.

Wiring it up

An Elasticsearch request that fuses a keyword leg and a kNN leg with RRF:

{
  "retriever": {
    "rrf": {
      "retrievers": [
        { "standard": { "query": { "match": { "text": "error 0x80070057 backup job" } } } },
        { "knn": { "field": "vector", "query_vector": [0.12, -0.03, 0.44], "k": 50 } }
      ],
      "rank_window_size": 50,
      "rank_constant": 60
    }
  }
}

The same shape in Qdrant, where prefetch runs each leg and the outer query fuses them:

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://localhost:6333")
client.query_points(
    collection_name="docs",
    prefetch=[
        models.Prefetch(query=models.SparseVector(indices=[1, 42], values=[0.22, 0.8]), using="sparse", limit=50),
        models.Prefetch(query=[0.12, -0.03, 0.44], using="dense", limit=50),
    ],
    query=models.RrfQuery(rrf=models.Rrf()),
    limit=10,
)

And the evaluation loop that decides whether any of it helped:

def recall_at_k(results: dict[str, list[str]], relevant: dict[str, set[str]], k: int) -> float:
    hits = sum(1 for q, ids in results.items() if relevant[q] & set(ids[:k]))
    return hits / len(results)

for name, runs in {"bm25": bm25_runs, "dense": dense_runs, "hybrid": hybrid_runs}.items():
    print(name, round(recall_at_k(runs, golden, k=20), 3))

What you’ll see

Healthy: fused recall@k at or above the best single leg across the set, with per-query wins split by type, identifier-heavy queries won by BM25 and paraphrase queries by the dense leg. Latency: p99 becomes the slower leg plus the merge, so budget for both scans.

Two bad shapes are diagnostic. Fused recall below the dense leg alone means the keyword leg is injecting junk: a stopword-heavy query against a badly configured analyzer, or a fusion window too small. Fused recall identical to the dense leg means the sparse leg is being zeroed out, usually alpha too high or min-max normalization on a leg that returned one result, which normalizes to a constant. If both legs are bad before fusion, the fault is upstream; work through low vector search recall causes and fixes first.

Caveats

• Fusion windows hide candidates. RRF and score fusion only see the top N of each leg. A document outside the window in one leg gets only the other leg’s contribution.
• Filters must apply to both legs before fusion. Filtering after the merge leaves holes at the top of the list; Qdrant’s prefetch and Elasticsearch’s retriever tree both take per-leg filters.
• The BM25 leg is only as good as its tokenizer. Hyphenated part numbers, identifiers with underscores and CJK text all behave differently under a default analyzer. Check what the index stored before blaming fusion.
• Learned sparse is a third option, not a BM25 synonym. SPLADE-style models produce sparse vectors that keep exact matching and inverted-index efficiency while adding term expansion. They change the sparse leg’s failure modes, so re-run the eval.
• Tuning alpha on the queries you report on is label leakage. Hold out a split; an alpha tuned on the test set flatters the number. The regression-baseline discipline in SentryML’s guide to LLM evals applies unchanged to retrieval.
• Retrieval surfaces are injection surfaces. The BM25 leg retrieves on literal token overlap, which is exactly what a planted document can be written to maximize. AI Sec’s notes on indirect injection in RAG pipelines cover the sanitization side.

For how the three main open-source engines expose these knobs, including which fusion each defaults to, see Qdrant vs Weaviate vs Milvus.

Related across the network

• Data Poisoning in RAG Systems: A 2026 Threat Briefing — ai-alert.org
• RAG Poisoning: How Retrieval-Augmented Systems Get Compromised — ai-alert.org
• Best Vector Database for RAG: A Practical Comparison (2026) — llmops.report
• Embedding Store Reliability: What to Monitor Beyond Recall@k — mlmonitoring.report
• MetInfo CMS CVE-2026-29014 Exploited in the Wild for RCE — ai-alert.org