Lexical, Dense and Hybrid Retrieval

Two queries, two failures

Harbourline's help-centre assistant gets two questions on the same afternoon. One is "form HO-3 water backup". The other is "is my sump pump covered". The first names a form and a peril. The second names neither. A dense retriever tuned on paraphrase handles the second well and often misses the first, because an identifier such as HO-3 is a few word pieces to an embedding model and sits near every other homeowner form. A lexical retriever does the reverse: it finds the exact string in the first query and finds nothing for "sump pump" if the wording says "sewer or drain backup".

This chapter is about running both, fusing the lists, and the index that makes the dense half fast. It builds on the embedding chapter, which covers what the vectors are, and it feeds reranking, which re-scores whatever this stage returns. The target is the coverage clause of the retrieval contract: the answering passage must be in the candidate set at all.

What BM25 computes

BM25 scores a document against each query term and adds the scores. For a term t in document d:

Three effects follow from the formula, and each predicts behaviour. First, idf makes rare terms dominate. "HO-3" appears in a fraction of documents, so it carries most of the score. "Water" appears everywhere and carries little. Second, saturation caps the reward for repetition: the tenth occurrence of a term adds far less than the second, so a long wording that says "backup" forty times does not beat a short one that says it three times by a factor of thirteen. Third, length normalisation penalises long documents, and b sets how hard. At b = 0 a 300-page wording wins on raw term counts. At b = 1 short procedure pages win. Chunks are more uniform in length than whole documents, which is one reason BM25 over chunks behaves better than BM25 over files. The formula is the standard form from Robertson and Zaragoza's 2009 survey. The defaults above are the common ones in Lucene-family engines, but check your engine.

BM25 wins on any query where the user's words are the document's words: policy numbers, form codes, endorsement names, a quoted phrase from a letter. It loses when the vocabulary differs, and it ignores word order and meaning entirely. It also depends on analysis: if your tokenizer splits HO-3 into ho and 3, the identifier match degrades into two common tokens. Test the tokenizer on your identifiers before you tune anything else.

What dense retrieval computes

A bi-encoder maps the query and each chunk to vectors independently, and retrieval is the nearest-neighbour search over those vectors. The chunk vectors are computed once at index time. Only the query is embedded at request time. That asymmetry is why it scales, and it is also its limit: the model compresses a whole chunk into one point before it has seen the query, so detail that does not move the point is lost. Numbers, negation, and exact identifiers are the usual casualties, as the embedding chapter explains.

Dense retrieval wins on intent and paraphrase. "Is my sump pump covered" lands near the passage on "sewer or drain backup" because training pushed those phrasings together. It also wins across languages and across the gap between customer wording and legal wording, which is wide in insurance.

The two methods fail on different queries, which is the evidence for running both. The BEIR benchmark, which tests retrievers zero-shot across a range of datasets, found BM25 a robust baseline that dense models often did not beat out of domain (Thakur et al., 2021). Your own evidence is the per-slice table from the labelled set: compute recall@20 for BM25 and for dense on each slice. Expect lexical to lead on the identifiers slice and dense to lead on definitions and procedures. If one method wins every slice by a wide margin, hybrid buys little and you should say so and skip it.

Fusing the two lists

You now hold two ranked lists with scores on incompatible scales. A BM25 score is unbounded and depends on the query's length and the corpus. A cosine similarity lives between roughly 0.2 and 0.9 and clusters tightly. Adding them directly lets whichever has the larger spread dominate.

Reciprocal Rank Fusion sidesteps the problem by ignoring scores. Each list contributes 1 / (k + rank) for each document, ranks start at 1, and a document absent from a list contributes nothing. Documents are sorted by the sum (Cormack, Clarke and Buettcher, 2009). A chunk ranked 1st by BM25 and 5th by dense scores 1/61 + 1/65. A chunk ranked 1st in one list and absent from the other scores 1/61, so agreement between lists beats a single strong vote, but only narrowly.

def rrf(lists, k=60, top=20):
    """lists: ranked lists of chunk ids, best first."""
    score = {}
    for ranked in lists:
        for rank, cid in enumerate(ranked, start=1):
            score[cid] = score.get(cid, 0.0) + 1.0 / (k + rank)
    return sorted(score, key=score.get, reverse=True)[:top]

bm25_hits  = ["c7", "c2", "c9", "c4"]
dense_hits = ["c2", "c5", "c7", "c1"]
print(rrf([bm25_hits, dense_hits]))   # c2 and c7 lead: both lists found them

The constant k sets how fast the reward decays with rank. Small k makes rank 1 dominate. Large k flattens the curve so deep ranks still count. The paper settled on k = 60 from a preliminary investigation, and Elasticsearch ships 60 as its default rank_constant (as of November 2026). Treat 60 as a sound start, then sweep 10, 30, 60 and 100 on the labelled set. Only the candidate depth feeds in, so the window matters as much as k: fusing two top-10 lists gives the fusion at most 20 candidates to work with, and a chunk at dense rank 25 cannot be rescued. Elasticsearch calls this rank_window_size. Fuse from 50 to 100 per list, then cut.

Weighted score fusion normalises each list, for example min-max over the returned scores, then takes w * lexical + (1 - w) * dense. It can beat RRF when you have enough labels to fit w, because it keeps the information that the top dense hit is far ahead of the second. It costs you: min-max over a top-100 list shifts with every query, and a weight tuned on one slice can hurt another. Pick RRF to start. Pick weighted fusion when the sweep shows a gain that survives on every slice, or when you need a query-type switch, such as weighting lexical higher when the query contains a form code. That switch is a legitimate use of the weights, because the identifiers slice is where lexical earns its place.

On where to run it: one engine that does both (Elasticsearch, OpenSearch, Weaviate, Qdrant and Postgres with pgvector plus full-text search can all return a lexical and a vector list) keeps one copy of the data, one set of filters and one consistency story. Two engines let you pick the best of each and cost you a sync pipeline and fusion in application code. The flip: if you already operate a mature lexical cluster and the vector side is new, keeping both is cheaper than migrating. If the corpus is 300k chunks, one engine is almost always right.

Filters and the ANN index

Harbourline's filters are not optional. A question about a UK policy must not return a US wording, and a question about a 2025 policy should prefer the 2025 version. These are metadata filters on jurisdiction, product and effective_date. Chapter 08 covers them as access control. Here the point is how they interact with the index.

Post-filtering runs the approximate search first, then drops results that fail the filter. If the filter keeps 10% of chunks and the search returns 40 candidates, about 4 survive. The pgvector documentation gives exactly this example with its default search width of 40 (as of November 2026). Pre-filtering restricts the search to matching chunks first. It returns a full list, but the graph traversal must route around excluded nodes, and engines handle this differently: some switch to an exact scan when the filter is selective, some apply the filter during traversal. The result for you is a rule: the more selective the filter, the more the engine's strategy matters, and the more you must test recall with the filter on. A recall number measured on unfiltered queries says nothing about production, where every query is filtered.

Two defences. In pgvector, since version 0.8.0, iterative index scans (hnsw.iterative_scan) keep scanning until enough rows pass the filter or a tuple limit (20,000 by default) is hit. In any engine, a highly selective filter such as one tenant out of thousands is better served by a partition or a separate index than by a filter.

Index choices and their cost

Exact (flat) search compares the query to every vector. At the roughly 667,000 chunks of 1,024 dimensions that chapter 03 arrives at for Harbourline, that is 667,000 dot products per query, about 2.7 GB of float32 vectors to scan. It is simple, exact and often fast enough below a few hundred thousand vectors. Do not add an approximate index until a measured latency says you need one, because the index trades recall for speed and the trade is silent.

Past that size you choose between two families. HNSW (Malkov and Yashunin, 2016) builds a layered graph. A query enters at the sparse top layer and descends, greedily moving to closer neighbours, and the authors report logarithmic scaling of search cost. Three parameters matter. M is the number of links per node: higher gives better recall and a bigger graph. ef_construction is the candidate list while building: higher gives a better graph and a slower build. ef_search is the candidate list at query time, and it is the dial you turn in production: higher raises recall and latency together. pgvector's defaults are m = 16, ef_construction = 64 and ef_search = 40 (as of November 2026). IVF clusters the vectors, then searches only the nearest nprobe clusters. It builds faster and uses less memory than HNSW, but recall depends on whether the query's neighbours sit across a cluster boundary, and it needs training data representative of the corpus.

Measure index recall against the flat result for the same queries: of the true top-20, how many does the index return? That number is separate from retrieval quality and is the one that ef_search moves. Pick the smallest ef_search that gets index recall above 0.98 under your real filters, then spend the rest of your latency budget on reranking.

Postgres with pgvector versus a dedicated store is mostly an operations question at this scale. pgvector indexes up to 2,000 dimensions for vector and 4,000 for halfvec (as of November 2026), keeps vectors beside your relational metadata so filters and joins are native, and gives you one backup story. A dedicated store earns its place when you need very large corpora, built-in hybrid search with managed fusion, or throughput that would contend with your transactional load. For roughly 12k documents, start in the database you already run, and move when a measurement, not a roadmap slide, says to.

✓ Check yourself

Answer from memory before opening each one.

1. A customer asks about "endorsement HO 04 61". Dense retrieval returns three scheduled-property endorsements with other numbers. Why does BM25 get this right, and what would break it even though the string is in the corpus?

   Show answer

   The rare token carries a high idf, so a chunk containing it outscores chunks with only common words, while the embedding places all endorsement text close together. It breaks if the analyser splits the code into common fragments such as "ho", "04" and "61", which destroys the rarity. See What BM25 computes.

2. You fuse a BM25 top-10 and a dense top-10 with RRF and recall@10 of the fused list is lower than dense alone. Name two causes you would check first.

   Show answer

   Candidate depth is too shallow, so documents ranked 11 to 50 in one list cannot be promoted, and the lexical list is mostly noise on queries without identifiers, where its votes displace good dense hits. Deepen both lists to 50 to 100 and test a query-type weight. See Fusing the two lists.

3. A teammate proposes min-max normalising both scores and adding them with weight 0.5 "because it uses more information than ranks". What do you ask before agreeing?

   Show answer

   Ask whether the gain holds on every slice and not just the average, and how stable the normalisation is, since min-max over a top-100 list shifts per query. If the sweep shows no gain, RRF wins on simplicity. If one slice improves and another regresses, the weight should depend on query type.

4. Recall@20 on the labelled set is 0.91 unfiltered and 0.74 once the jurisdiction filter is applied, with the same index. Where is the loss, and what do you change?

   Show answer

   The approximate search is returning too few candidates that pass the filter: with post-filtering, the filter discards most of a small candidate list. Raise the search width, turn on iterative scanning or pre-filtering, or partition by jurisdiction. See Filters and the ANN index.

5. The corpus is 12k documents and p95 latency is 40 ms with an exact scan. A colleague wants HNSW "for scale". Make the judgement.

   Show answer

   Keep the exact scan: it has no recall loss and the latency is well inside budget. HNSW wins when the scan crosses your latency budget, or the corpus grows by an order of magnitude. Until then the index adds a tuning surface and a silent recall loss. The flip condition is measured latency, not anticipated size.

Takeaway

• BM25 wins when the user's words are the document's words, and dense wins when they are not, so the identifiers slice and the definitions slice should show opposite leaders.
• Tokenisation decides whether an identifier is a rare term or two common ones. Test it on form and policy numbers first.
• RRF fuses ranks, not scores, so it needs no calibration. Start at k = 60, fuse from 50 to 100 candidates per list, and sweep k on your set.
• Weighted fusion earns its place only when the gain holds on every slice, or when the weight switches by query type.
• Measure recall with the production filters switched on. Post-filtering silently shrinks the candidate list.
• Below a few hundred thousand vectors, an exact scan is a respectable default. Above it, tune ef_search against the exact result before touching anything else.

Fusion gets the right chunk into the candidate set. Getting it to the top of that set is a different tool. → Reranking

Further reading

• Reciprocal Rank Fusion outperforms Condorcet and individual rank learning methods — Cormack, Clarke and Buettcher, 2009. Two pages; the source of the formula and k = 60.
• BEIR — Thakur et al., 2021. Evidence that BM25 is a strong out-of-domain baseline.
• Efficient and robust approximate nearest neighbor search using HNSW graphs — Malkov and Yashunin, 2016. The graph structure behind M and ef.
• pgvector README — HNSW and IVFFlat options, defaults, and iterative scans for filtered queries.
• Elasticsearch reciprocal rank fusion — the formula as implemented, with rank_constant and rank_window_size.

For the interview framing of the same material, see Hybrid search and reranking, and for how this stage sits in a regulated deployment, RAG applied.