BM25 — Deep Dive

Imagine a librarian who memorizes every exact word on every book’s spine but never reads the contents. That is BM25 — a search system that finds documents by matching the exact words you type, nothing else. It exists so you can retrieve information when you know the precise terms, not just the general idea, without needing expensive computers or training.

Now picture that librarian building a giant cabinet of index cards — one card for each unique word, listing every book that contains it. When you ask for “error codes,” the librarian only opens the cards for “error” and “codes,” counts how many books hold each word, and ranks them by rarity (rare words matter more) and repetition (a word showing up many times in one book gets a bonus but saturates, controlled by the k1 parameter). The library doesn’t run any fancy software — just those paper cards (the inverted index) and two knobs: k1 for how much repetition helps, and b for whether long books are unfairly rewarded. Every choice is transparent: you can see exactly why “codes” beat “code” because the librarian only matched the exact spelling.

The trickiest detail is that the librarian first decides what counts as a “word.” For a technical book like yours, QueryFusionRetriever could be left as one massive word or split into query, fusion, retriever. Split it, and the librarian now also matches “query fusion” — but loses the ability to find that exact string exclusively. The code even has a guard: if you ask for ten matches but only three books exist, the librarian clamps to three rather than crashing. Without this exact‑word system, you would miss precise identifiers like error messages or API names — the dense‑retrieval librarian who understands meaning would lump similarity_top_k with other “similar things,” leaving you empty‑handed when you typed the exact version.

BM25 provides a training-free, interpretable sparse retriever, built on demand from Qdrant payloads with a safety clamp for small indices.

retrievers = [index.as_retriever(similarity_top_k=retrieve_k)]   # dense leg, top-20
nodes = store_nodes(index)   # hydrate corpus text back out of Qdrant payloads
if nodes:
    # bm25s errors if top_k > corpus size — clamp for small indices.
    retrievers.append(BM25Retriever.from_defaults(
        nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))))
else:
    print("[hybrid] no nodes in store — dense-only fusion")

The subsystem is a hybrid retrieval engine built on a shared Qdrant store of roughly 22,000 node points. The ordered mechanism begins when a query enters the LOOK router, which selects an already‑constructed engine from a process‑lifetime cache, ensuring BM25 hydration occurs only once per strategy. First, the dense leg (via index.as_retriever(similarity_top_k=retrieve_k)) retrieves its top‑20 candidates. Second, store_nodes(index) hydrates the full corpus text out of Qdrant payloads. If nodes exist, a BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))) is built—the clamp prevents the bm25s backend from erroring when the corpus is smaller than the requested top‑k. If hydration returns nothing or the BM25 package is unavailable, the engine degrades to dense‑only fusion with a log line [hybrid] no nodes in store — dense-only fusion. Both legs then enter QueryFusionRetriever in reciprocal_rerank mode with num_queries=1, fusing by rank. The fused candidates pass through node postprocessors in deliberate order: injection fencing first, cross‑encoder reranker second. On failure (e.g., BM25 completely unavailable), the service logs that line and continues with dense only, never breaking the query path.

The invariant the design preserves is single source of truth—the node texts in Qdrant payloads are the only copy of the corpus. Re‑ingesting the corpus updates both the dense vectors and the sparse leg together: “there is no second index to drift out of sync.” This guaranties that neither leg can become stale relative to the other, avoiding silent recall loss from index divergence. The sparse leg is reconstructed from the same payloads on every process restart (but memoized per engine), so the two legs always share the same textual representation.

The key trade‑off is corpus‑in‑RAM simplicity versus memory cost. The in‑process BM25 architecture avoids maintaining a second separate index, which would drift out of sync with the dense store, but it requires holding all 22k hydrated node texts in process memory. The obvious alternative it rejects is “pushing the sparse leg server‑side as Qdrant native sparse vectors,” which would add a second representation to maintain and keep in sync across re‑ingestions. That rejection avoids the operational burden of index drift and the complexity of two distinct indexing pipelines. The cost is that on memory‑capped production hosts (a 512 Mi free‑tier container), the sparse leg must be shed entirely—HYBRID is switched off—so queries that rely on exact identifiers and API names lose coverage.

One concrete failure mode is the bm25s top‑k error. If the corpus size is smaller than the requested similarity_top_k and the clamp were omitted, bm25s raises an error (“errors if asked for top_k larger than the corpus”). The signal an operator would see is that error log from the bm25s library, causing the entire hybrid engine build to fail unless the defensive min(retrieve_k, len(nodes)) is in place. With the clamp, a different failure is when store_nodes(index) returns an empty list—perhaps because the Qdrant docstore is empty or the payloads were not hydrated—the operator sees the log line [hybrid] no nodes in store — dense-only fusion, indicating the sparse leg is silently dropped.

Failure 1: Corpus Size Smaller than Requested similarity_top_k

• Trigger The BM25 backend bm25s raises an error when similarity_top_k exceeds the number of nodes in the corpus. This occurs when retrieve_k > len(nodes), which happens during small test indexes or corpus shrinkage.
• Guard min(retrieve_k, len(nodes)) inside the call to BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))). This clamp prevents the backend error by capping the requested top‑k to the available documents.
• Posture Fail‑soft – the retriever returns fewer results than originally configured, but the query path completes without aborting.
• Operator signal No log line or metric; the clamp is silent. The operator may observe that the number of results returned is less than the intended similarity_top_k, but no error is raised.
• Recovery Automatic – the clamp applies every time the hybrid engine is built.

Failure 2: Node Hydration Returns Empty List

• Trigger The function store_nodes(index) fails to extract node texts from Qdrant point payloads (e.g., empty store, missing text field, or network issue), returning an empty list [].
• Guard The conditional if nodes: check; when nodes is falsy, the engine prints "[hybrid] no nodes in store — dense-only fusion" and skips appending the BM25 retriever. The else branch then proceeds with dense‑only fusion.
• Posture Fail‑soft – the sparse leg is dropped, and retrieval degrades to dense‑only without crashing.
• Operator signal The exact log line [hybrid] no nodes in store — dense-only fusion appears in the process output.
• Recovery No retry. The system continues with the dense‑only fallback. Manual investigation of Qdrant payloads or the store_nodes implementation is needed to restore the BM25 leg.

Failure 3: Stale Corpus Statistics (IDF and avgdl)

• Trigger Large additions, deletions, or modifications to the corpus occur after the BM25 retriever was built. IDF and avgdl are baked at index construction time and are not updated online.
• Guard None shown in the source. The context notes that “the sparse index must be rebuilt, and an in‑process BM25 leg must be re‑hydrated” but provides no exception handler, validation, or automatic refresh.
• Posture Fail‑soft – BM25 scores gradually lose accuracy (outdated term weights), but the retrieval pipeline continues to run.
• Operator signal No explicit warning or error; degradation is silent. Operators may only detect it through offline evaluation of recall or precision.
• Recovery Manual – rebuild the BM25 leg by restarting the process or triggering a re‑hydration of nodes from Qdrant (the hybrid engine rebuilds on each start, but not while the process is live).

Failure 4: Fusion Dilution from a Weak Sparse Leg

• Trigger The sparse leg is weak (e.g., tiny corpus, poor analyzer) and injects noisy rankings into the hybrid fuse. The QueryFusionRetriever with mode="reciprocal_rank" and num_queries=1 weights both legs equally via RRF, so the weak leg can dilute a strong dense ranking.
• Guard No explicit guard exists in the source. The context warns that “measured pipelines have seen hybrid underperform the best standalone leg when signals diverge” but provides no detection or fallback.
• Posture Fail‑soft – the hybrid retrieval continues, but its effectiveness may be worse than running the dense leg alone.
• Operator signal Subtle; no log or metric from the code. Requires A/B comparison or metric monitoring to notice.
• Recovery Manual – a operator must either remove the weak leg from the retrievers list, improve the sparse leg’s analyzer or corpus, or adjust fusion weights (not supported by the current setup).

Imagine you’re looking for a rare, specific tool in a huge toolbox. BM25 is like a smart assistant that knows the rare tools are the important ones, and it doesn’t get tricked by common items or by extra-long manuals. Its job is to rank which toolbox compartments (documents) are most relevant to your search.

The assistant first gives extra weight to uncommon tools—a term that appears in only a few compartments gets a big score boost (that’s inverse document frequency). Then it counts how many times the tool appears: the first few mentions matter a lot, but after that, extra copies add almost nothing (controlled by k1, the term‑frequency saturation parameter). Finally, it adjusts for how long the manual inside the compartment is, so a short manual that names the tool once beats a long one that mentions it many times accidentally (controlled by b, length normalization). Both k1 and b can be tuned.

The trickiest part is tuning them correctly. For short fields like titles, a low k1 works best because repeating a rare tool there is noise, not signal. When the corpus is already cut into fixed‑size chunks, length normalization (b) has little left to do, so a small b is defensible. Without this careful scoring, common words like “the” would overwhelm rare terms, and long, boilerplate documents would always win—so you’d never find the exact identifier or error message you were actually looking for.

The hybrid engine builds an in-process BM25 retriever from hydrated Qdrant payloads, clamping top_k to avoid bm25s errors on small corpora.

nodes = store_nodes(index)   # hydrate corpus text back out of Qdrant payloads
if nodes:
    retrievers.append(BM25Retriever.from_defaults(
        nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))))

The BM25 scoring subsystem operates as a deterministic, interpretable retrieval leg built on an inverted index of term→posting list, constructed once by tokenizing the entire corpus. On query, the mechanism proceeds through ordered steps: the query is tokenized to extract terms; for each term, its posting list is fetched from the index; the score for each document is computed as a sum over matching terms of IDF(t) * (f(t,d) * (k1+1)) / (f(t,d) + k1 * (1 - b + b * (|d|/avgdl))), where f(t,d) is term frequency, k1 controls saturation, b controls length normalization, and avgdl is the mean document length baked at index time. The documents are then ranked by this score. On failure—for example, if the query analyzer uses a different stemmer than the index analyzer—the term lookup silently returns no matches or partial matches, producing a recall loss that is invisible to the operator at query time because no error is thrown; scores are computed on whatever tokens survive, and the system continues.

The design preserves an explicit invariant: BM25 scores are not probabilities and not comparable across queries or corpora; never threshold them globally. Fuse by rank, not by raw score. This guarantee means the raw numeric output of BM25 is only meaningful within a single query’s ranking. The downstream pipeline (whether a single leg or fused with dense retrieval via Reciprocal Rank Fusion and a cross-encoder) must never treat the BM25 score as an absolute relevance signal. The system enforces this by construction—the RRF fusion function discards scores entirely and fuses ranks using RRF(d) = Σ 1/(k + rank_leg(d)), where k is a smoothing constant. This invariant is what allows BM25 to be combined with dense cosine similarities without calibration.

The key trade-off is accepting exact-term matching with no semantic understanding in exchange for a training-free, GPU-free, fully interpretable retrieval leg that transfers to any domain. The obvious alternative it rejects is a second dense embedding model that captures paraphrase and synonyms. That rejection avoids the cost of training or fine-tuning, generating embeddings for every document, and serving GPU inference per query. The source states: “BM25 has no training phase, no embedding model, no GPU … fast, cheap, fully interpretable … transfers to any domain without fine-tuning.” The trade-off is that BM25 cannot answer “how does the platform merge keyword and vector results” unless the query contains those exact words—hence the hybrid system adds a dense leg precisely because BM25 alone would miss semantically equivalent questions.

One concrete failure mode is analyzer mismatch between index and query time. Suppose the index tokenizer lowercases and stems the corpus (e.g., “running” → “run”), but the query tokenizer does not stem (e.g., “running” stays “running”). The posting list for “running” is empty, so the query gets zero matches or only matches from tokens that erroneously split. The operator would see a silent recall loss: the top-k results from BM25 suddenly contain no documents that contain the word “running,” even though hundreds exist, and the dense leg (if present) might rescue some, but the BM25 leg contributes nothing. The signal is not a crash or error log; it is a shift in the distribution of retrieved document IDs—queries with inflected forms, compound terms, or code identifiers will consistently underperform compared to exact-match queries, visible in a monitoring dashboard as a drop in Recall@k for queries containing any non-matching analyzer token.

Stale Corpus Statistics

• Trigger — Large changes to the corpus (additions, deletions, re-ingestions) after the BM25 index was built, so IDF and avgdl become incorrect for the current document population.
• Guard — None shown in the source. The text notes that “the sparse index must be rebuilt, and an in-process BM25 leg must be re-hydrated,” but this is a manual maintenance step, not an automatic guard.
• Posture — fail-soft: the system continues to return results using the stale statistics, causing silent recall loss.
• Operator signal — No log line or metric is emitted; the degradation is invisible unless recall is measured against a ground-truth set.
• Recovery — Manual rebuild: re-run the hybrid engine construction so that BM25Retriever.from_defaults(nodes=nodes, ...) is built over the current corpus.

Analyzer Mismatch

• Trigger — The tokenizer, stemmer, or splitter used at query time differs from the one used when the BM25 index was built (e.g., one pipeline applies stemming while another does not).
• Guard — None shown in the source. The failure is only described as a known risk (“silent recall loss on stemmed/split terms”).
• Posture — fail-soft: queries continue to execute, but terms that should match are missed, silently reducing recall.
• Operator signal — No error or warning; the operator would observe unexpectedly low overlap between dense and sparse results, or poor recall on queries containing inflected forms.
• Recovery — Audit and align the tokenization chain across all code paths that feed into BM25Retriever and the indexed corpus.

Small‑Corpus Top‑k Clamp

• Trigger — The corpus size (len(nodes)) is smaller than retrieve_k, which causes the bm25s backend to raise an error. This is most likely in test deployments or early stages with very few nodes.
• Guard — similarity_top_k=min(retrieve_k, len(nodes)) in the line BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))).
• Posture — fail-soft: the value is silently clamped to the corpus size, so the BM25 leg retrieves fewer candidates than intended but does not crash.
• Operator signal — No log line is produced; the operator may notice that the BM25 leg returns fewer results than retrieve_k only by inspecting the returned list lengths.
• Recovery — Automatic: the clamp ensures the query succeeds. No operator action required, though the reduced top‑k may slightly affect fusion quality.

BM25 Hydration Failure

• Trigger — The hydration step store_nodes(index) returns an empty list (e.g., because Qdrant payloads lack the text field, or the store is empty), or the bm25s package is not installed/importable.
• Guard — The conditional if nodes: followed by else: print("[hybrid] no nodes in store — dense-only fusion"). The fallback skips the BM25 leg entirely.
• Posture — fail-soft: the hybrid engine degrades to dense‑only fusion, logging the event but continuing to serve queries.
• Operator signal — The exact log line [hybrid] no nodes in store — dense-only fusion printed to stdout (or captured by the logging system).
• Recovery — Automatic: the dense leg takes over; no retry or manual step required. To restore the sparse leg, the operator must either populate the payloads or ensure the bm25s package is installed and restart the service.

Fusion Dilution

• Trigger — The sparse leg is weak (tiny corpus, poor analyzer, stale statistics) and injects low‑quality results into the reciprocal‑rank fusion, causing the hybrid ranking to underperform the best standalone leg.
• Guard — None shown in the source. The text warns that “measured pipelines have seen hybrid underperform the best standalone leg when signals diverge,” but no automatic guard (e.g., a quality threshold or fallback) is implemented.
• Posture — fail-soft: the system continues to fuse and return the diluted ranking; there is no crash or fallback.
• Operator signal — No immediate error. The operator would observe a regression in retrieval metrics (recall, NDCG) or an increase in user complaints that results are worse than before.
• Recovery — Manual: inspect the sparse leg’s corpus size, analyzer configuration, and statistics; either improve it or disable the BM25 leg entirely. The code path to disable is not shown in the source.

Score Misuse

• Trigger — An operator (or downstream system) treats the BM25 score as a probability, applying a global threshold to filter results, or relies on the raw score for comparisons across queries.
• Guard — None shown in the source. The text states “BM25 scores are not probabilities and not comparable across queries or corpora; never threshold them globally. Fuse by rank, not by raw score” — but this is advisory, not guarded by code.
• Posture — fail-soft: the system produces the raw scores, but if thresholding is applied externally, incorrect filtering or ranking occurs. The scoring and fusion logic itself (reciprocal_rerank mode) correctly uses ranks, so the failure is in the calling layer.
• Operator signal — No signal from the BM25 subsystem; the operator may see inconsistent result quality or empty result sets when a threshold is too high.
• Recovery — Manual: remove any global score thresholds and rely on rank‑based fusion as implemented by QueryFusionRetriever with mode="reciprocal_rerank".

Imagine you’re looking for a recipe that uses “baking soda” but the cookbook only indexes by meaning—it would give you “leavening agent” instead. This system’s BM25 leg is like a librarian who scans every page for the exact words you say, while the dense leg is a librarian who grasps concepts. BM25 exists to catch those precise identifiers—version numbers, error messages, or class names—that dense retrieval blurs into close-but-wrong matches.

To do this, BM25 breaks your query into tokens, then scores each document by how many times those tokens appear, weighted by how rare they are. The code builds a BM25Retriever from the same corpus of 22,000 nodes stored in Qdrant, using two knobs: k1 (how much repeated terms matter) and b (how much document length influences the score). When you search for an exact API parameter like similarity_top_k, BM25 finds the document with that exact string, while dense retrieval would return everything about “query parameters.”

The non‑obvious edge is a defensive guard: if the corpus is tiny, the code clamps similarity_top_k to the number of nodes (min(retrieve_k, len(nodes))) because the bm25s backend crashes if you ask for more results than documents exist. Without BM25, a beginner searching for an error string like “bm25s errors if top_k > corpus size” would get only semantically related text—never the precise troubleshooting page. Your search feels broken because the one exact match you needed is silently erased.

The hybrid retrieval engine builds an in-process BM25 retriever from the same Qdrant payloads, keeping a single source of truth and costing no extra infrastructure.

retrievers = [index.as_retriever(similarity_top_k=retrieve_k)]   # dense leg
nodes = store_nodes(index)   # hydrate corpus text from Qdrant payloads
if nodes:
    # bm25s errors if top_k > corpus size — clamp for small indices.
    retrievers.append(BM25Retriever.from_defaults(
        nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))))
else:
    # degrade to dense-only fusion
    pass

The subsystem’s ordered mechanism begins with the dense leg: index.as_retriever(similarity_top_k=retrieve_k) fetches the top-20 from Qdrant. Next, the corpus text is hydrated from the same Qdrant point payloads via store_nodes(index). If nodes exist, a BM25Retriever is constructed with similarity_top_k=min(retrieve_k, len(nodes)) to avoid a bm25s backend error when the corpus is smaller than the requested top-k. On failure—if hydration returns no nodes or the BM25 package is unavailable—the engine logs a line and degrades to dense-only fusion. Both retrievers are then fused by QueryFusionRetriever in reciprocal_rerank mode with num_queries=1. Finally, the fused candidate set passes through a cross-encoder reranker, which re‑scores and truncates to the final top-k; the fusion cut is widened to the reranker’s budget first, so the reranker has meaningful candidates to demote.

The design preserves the invariant of single source of truth: the corpus lives only in Qdrant point payloads, and the BM25 leg is built fresh from those payloads at engine construction. There is no second, separately‑ingested sparse index that could drift out of sync with the dense store. Re‑ingesting the corpus updates both legs atomically, and the service memoizes the built engine per resolved strategy so that hydration (an O(corpus) operation) happens once per process per strategy, not once per query.

The key trade‑off is that the in‑process BM25 architecture buys “single‑source‑of‑truth simplicity at the price of corpus‑in‑RAM.” The obvious alternative it rejects is pushing the sparse leg server‑side as Qdrant native sparse vectors, which would require a second representation to maintain in the collection and introduce the cost of index drift and sync complexity. Instead, the system keeps a single representation in Qdrant and hydrates text into memory only when needed. On the memory‑capped production host (512 MiB), this trade‑off forces the hybrid profile off entirely—the sparse leg is shed first because holding 22k hydrated node texts does not fit next to the service itself. Queries that suffer are exactly those the sparse leg exists for: exact identifiers, API names, error strings.

One concrete failure mode is analyzer mismatch between index and query time, which silently loses recall on stemmed or split terms. An operator would see the signal of high overlap statistics between the two legs’ top‑20 results: when overlap is large, the sparse leg is redundant and not adding coverage, indicating that the analyzer is aligning poorly with the dense leg’s semantic space. Conversely, low overlap on queries known to contain exact tokens (e.g., mode="reciprocal_rerank") suggests the sparse leg is under‑performing due to stale corpus statistics or a mismatched tokenizer, a signal visible in per‑query recall logs.

BM25 vs Dense: Failure-Mode Analysis

1. bm25s top_k Exceeds Corpus Size

• Trigger — similarity_top_k set to a value larger than len(nodes) (the number of documents in the corpus). This occurs in small test indices or after a corpus purge.
• Guard — similarity_top_k=min(retrieve_k, len(nodes)) clamps the requested depth to the actual corpus size.
• Posture — fail-soft. The BM25 leg continues with a reduced top_k, and the fusion step still runs.
• Operator signal — No explicit log line; the operation is silent. The operator could infer the clamp by noticing that the fused output contains fewer than retrieve_k items when the corpus is small, but no error is raised.
• Recovery — Automatic. The clamped value is used for that query. No retry; the system simply returns fewer results.

2. Hydration Returns No Nodes from Qdrant

• Trigger — store_nodes(index) returns an empty list or None when the Qdrant store contains no points, its payloads are missing text, or the docstore is empty on the Qdrant path.
• Guard — The if nodes: conditional: if nodes: retrievers.append(BM25Retriever(...)) else print("[hybrid] no nodes in store — dense-only fusion").
• Posture — fail-soft. The engine degrades to dense-only fusion, serving a response from the dense leg alone.
• Operator signal — The exact log line: "[hybrid] no nodes in store — dense-only fusion". Printed to stdout/stderr.
• Recovery — Immediate fallback to dense-only. No retry; the operator must inspect the Qdrant store to repopulate nodes or fix the hydration path.

3. Fusion Dilution from a Weak Sparse Leg

• Trigger — The BM25 leg operates on a tiny corpus (e.g., after partial reindexing) or with a misconfigured analyzer, so it returns noisy or irrelevant top‑20 results. RRF equally weights this weak leg against a strong dense leg, injecting noise and degrading overall ranking.
• Guard — No explicit guard shown in the source. The system does not measure overlap statistics or compare leg quality before fusion.
• Posture — fail-soft but silent. The hybrid ranking may underperform the dense leg alone, but the system continues to return results.
• Operator signal — A drop in downstream metrics (e.g., recall, user satisfaction) is observable only if monitored. No runtime log or error is emitted.
• Recovery — Manual. The operator must inspect overlap statistics, rebuild the sparse index, or adjust fusion weights (the source notes “RRF weights legs equally by construction”).

4. Analyzer Mismatch Between Index and Query Time

• Trigger — The tokenization/stemming settings used at BM25 indexing time differ from those applied at query time (e.g., using a Porter stemmer during building but not during query). This causes silent recall loss on stemmed or split terms.
• Guard — No guard is shown in the source. The system does not validate that the analyzer configuration is consistent.
• Posture — fail-soft and silent. BM25 returns fewer relevant documents, but no error is raised.
• Operator signal — The BM25 leg retrieves fewer or irrelevant results compared to the dense leg, observable via overlap statistics (“Overlap statistics between the legs … high overlap means redundancy, low overlap means coverage”). No direct log line indicates the mismatch.
• Recovery — Manual. The operator must ensure the same tokenizer settings are used during corpus hydration and query scoring.

5. Stale Corpus Statistics (IDF, avgdl) After Large Corpus Changes

• Trigger — The corpus is ingested once, baking IDF and average document length (avgdl) into the BM25 index. After significant additions, deletions, or content changes, these statistics become inaccurate, degrading BM25 scoring quality.
• Guard — No guard shown. The source explicitly states “after large corpus changes the sparse index must be rebuilt, and an in-process BM25 leg must be re‑hydrated.” The system does not detect staleness automatically.
• Posture — fail-soft. BM25 continues to produce scores based on outdated statistics, potentially returning worse rankings.
• Operator signal — A gradual quality drop in the sparse leg’s results (observable via increased overlap with dense leg or decreased recall). No explicit runtime signal.
• Recovery — Manual. The operator must rebuild the BM25 leg by re‑hydrating the nodes from Qdrant (the nodes = store_nodes(index) path) and reconstructing the BM25Retriever.

6. BM25 Package Unavailable at Import or Runtime

• Trigger — The bm25s (or rank_bm25) Python package is missing, wrong version, or fails to load when BM25Retriever.from_defaults(...) is called.
• Guard — No explicit guard is shown in the provided snippet. The source only says the engine degrades “if the BM25 package is unavailable,” but the code does not catch an ImportError or ModuleNotFoundError; it simply prints the hydrate‑failure message if nodes are empty. Without a try‑except, an import error would raise an unhandled exception.
• Posture — fail-hard (likely) if the import fails before the if nodes: check, or fail-soft if the error is caught elsewhere (not shown). Based on the source, the intended behavior is soft (degrade to dense-only), but the actual code as shown does not implement that guard.
• Operator signal — A ModuleNotFoundError traceback or, if a guard exists outside the excerpt, the log line "[hybrid] no nodes in store — dense-only fusion" (since the code conflates “no nodes” with “package unavailable”).
• Recovery — In the absence of a shown guard, the process terminates with an unhandled exception. If a guard were present, the engine would fall back to dense-only as described.

Imagine two friends searching for a lost item in a messy room. One friend looks only for exact labels on boxes, the other guesses based on what the item looks like and similar shapes. Each friend brings you their top twenty guesses, but their confidence scores are in different units, so you cannot simply add them together. This system blends their two ranked lists into one final list, giving you the best from both searchers. The two friends are called retrieval legs: one uses exact term matching, the other uses meaning-based matching. Both retrieve their top twenty candidates, but because their raw scores are not comparable, the system uses Reciprocal Rank Fusion, which focuses on rank positions. It applies a simple formula with a smoothing constant to give each candidate a blended score, so a box that ranks first in one list but tenth in the other still gets a fair chance. The trickiest detail is that these two legs often find very different sets of boxes: a medical study found only forty-two percent overlap between retrieval methods on the same queries. This means most candidates are unique to one friend, and the smoothing constant prevents the friend with higher typical scores from dominating the fusion, ensuring neither search method is silently ignored. Without this blending, you would have to pick one friend, losing all the items only the other friend could find. A beginner searching for "how to stop made-up answers" would never discover documents about "reducing hallucination" if only the exact-label friend was used, leaving you frustrated and empty-handed.

Setting up dense and BM25 retrievers for rank-based fusion via QueryFusionRetriever.

retrievers = [index.as_retriever(similarity_top_k=retrieve_k)]   # dense leg, top-20
nodes = store_nodes(index)   # hydrate corpus text back out of Qdrant payloads
if nodes:
    # bm25s errors if top_k > corpus size — clamp for small indices.
    retrievers.append(BM25Retriever.from_defaults(
        nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))))

The hybrid retrieval subsystem operates as a strict pipeline with explicit fallback paths. First, the dense leg—index.as_retriever(similarity_top_k=20)—retrieves the top‑20 candidates. Next, the corpus text is hydrated from Qdrant point payloads via store_nodes(index). If nodes exist, a BM25Retriever.from_defaults() is built with similarity_top_k clamped to min(retrieve_k, len(nodes)) to avoid the bm25s backend error when the corpus is smaller than the requested top‑k. If hydration returns nothing or the BM25 package is unavailable, the engine prints "[hybrid] no nodes in store — dense-only fusion" and degrades gracefully to a single‑leg fusion. The two result lists are then fused by QueryFusionRetriever in reciprocal_rerank mode with num_queries=1, producing a single fused candidate set. That candidate list passes through node postprocessors in a deliberate order: injection fencing first, then the cross‑encoder reranker second. When the cross‑encoder is enabled, the fusion cut is widened to the reranker’s candidate budget before truncating to the final top‑k.

The design preserves the single‑source‑of‑truth invariant: the corpus text is always extracted from Qdrant point payloads rather than maintained in a separate index. Re‑ingesting the corpus updates both legs simultaneously, and no second index can drift out of sync. This invariant is guaranteed by the hydration step—the BM25 leg is constructed in‑process over the same nodes that the dense leg queries, and the engine is memoized per resolved strategy so that hydration happens once per process per strategy, never per request.

The key trade‑off is simplicity of truth maintenance versus memory cost. The in‑process BM25 architecture avoids maintaining a second index representation (the obvious rejected alternative is pushing the sparse leg server‑side as Qdrant native sparse vectors). That alternative would introduce a second representation to keep consistent with the dense vectors, raising the risk of index drift and requiring additional operational overhead. The cost it avoids is the complexity of synchronising two separate indices. The price paid is that the entire corpus (~22k nodes) must be held in process memory; on memory‑constrained production hosts (512 MiB) this forces the HYBRID flag off, leaving only the dense leg running.

A concrete failure mode is the bm25s backend error when similarity_top_k exceeds the corpus size. An operator would see the clamp logic activate: the similarity_top_k passed to BM25Retriever.from_defaults() is truncated to len(nodes). If the corpus were empty, the log message "[hybrid] no nodes in store — dense-only fusion" would appear. Another signal is fusion dilution: when the sparse leg is weak (tiny corpus, bad analyzer), RRF weights legs equally, so a weak BM25 leg injects noise into a strong dense ranking. The operator would observe hybrid underperforming the best standalone dense retriever in measured pipelines, confirming that RRF is a recall strategy, not a precision strategy.

BM25 Top-K Exceeds Corpus Size

• Trigger — retrieve_k (e.g., 20) is greater than the total number of nodes in the Qdrant store, so constructing a BM25Retriever.from_defaults without clamping would raise an exception from the bm25s backend.
• Guard — The min(retrieve_k, len(nodes)) clamp applied to similarity_top_k inside the call to BM25Retriever.from_defaults.
• Posture — fail-soft: the sparse leg still runs but with an adjusted similarity_top_k equal to the corpus size, and fusion proceeds normally.
• Operator signal — Silent; no log line or error field is emitted. The operator would only notice the reduced top-k through separate monitoring of retrieval counts.
• Recovery — Automatic: the clamp activates every time the hybrid engine is built; no manual step required.

Node Hydration Returns Empty Corpus

• Trigger — store_nodes(index) returns an empty list (e.g., the Qdrant collection is empty, or payload extraction fails).
• Guard — The if nodes: branch; when it is False, the code executes print("[hybrid] no nodes in store — dense-only fusion").
• Posture — fail-soft: the dense leg runs alone; the sparse leg is omitted entirely.
• Operator signal — The exact log line [hybrid] no nodes in store — dense-only fusion is printed.
• Recovery — Automatic: the fusion engine degrades to dense-only for that query. To restore the sparse leg, the operator must populate or repair the Qdrant store and re-hydrate.

Stale Corpus Statistics After Large Corpus Changes

• Trigger — The corpus (Qdrant collection) grows or changes significantly after the BM25 leg was built, but the in-process BM25Retriever was constructed over old node texts; its IDF and avgdl are baked from the earlier token frequencies and are no longer accurate.
• Guard — No guard is shown in the source. The BM25 leg is rebuilt only when a new hybrid engine is instantiated; there is no automatic invalidation or periodic refresh.
• Posture — fail-soft (degraded ranking): the retrieval still runs, but term weights are stale, causing silent recall loss.
• Operator signal — Silent; no metric or log line is emitted. The operator would detect this only by comparing recall metrics over time or by manual inspection of IDF values.
• Recovery — Manual: the operator must rebuild the BM25 leg by re-running store_nodes(index) and constructing a fresh BM25Retriever.from_defaults(nodes=nodes, ...) with up-to-date node texts.

Analyzer Mismatch Between Index and Query Time

• Trigger — The tokenizer (e.g., stemmer, splitter) used by the BM25 retriever at query time differs from the one that produced the term frequencies stored in the bm25s index; for example, the index was built with a stop‑word list that is not applied at query time, or vice versa.
• Guard — No guard is shown in the source. The analysis pipeline is implicit in BM25Retriever.from_defaults and the underlying bm25s library; the context warns of the failure but provides no validation or exception.
• Posture — fail-soft: the engine runs but silently loses recall on stemmed or split terms.
• Operator signal — Silent; no error or log line. The effect is only observable through recall audits or mismatch in top‑k results.
• Recovery — Manual: the operator must ensure the same analyzer settings (tokenizer, stemmer, stop words) are used during index construction and query time.

Fusion Dilution From Weak Sparse Leg

• Trigger — The sparse leg (BM25) is weak—e.g., it covers a tiny corpus, uses a bad analyzer, or retrieves noisy results—so its top‑20 rankings inject irrelevant documents into the QueryFusionRetriever fusion with mode="reciprocal_rerank", potentially causing hybrid performance to underperform the dense leg alone.
• Guard — No guard is shown in the source. The QueryFusionRetriever weights legs equally by RRF construction; there is no mechanism to detect or discard a weak leg.
• Posture — fail-soft: the system continues to fuse both legs, but relevance may degrade.
• Operator signal — Silent; no error or log line. The operator would notice only through evaluation metrics (e.g., NDCG drop) or user feedback.
• Recovery — Manual: the operator must inspect the sparse leg’s configuration (corpus size, analyzer, IDF health) and either disable it or adjust the fusion weights (not shown in source) to reduce its influence.

Think of a detective who only finds clues by matching the exact words you give him—if you say “error code 404,” he ignores everything except that exact phrase. That’s the BM25 leg: its job is to catch precise, rare identifiers in a library of technical documents, while a second detective hunts by meaning.

Now, both detectives work from the same file room. Each document is stored as a point with two parts: a dense vector (for the meaning detective) and a payload holding the actual text. The exact-word detective does not keep his own separate card catalog. Instead, he reads the payloads from the same store, builds a temporary keyword index in his head (an in‑process retriever), and searches from there. This keeps everything in sync—no duplicate records to drift apart. The system also checks that the number of suspects he is asked to find (the similarity_top_k parameter) never exceeds the actual number of files; the bm25s package would throw an error if you asked for more results than documents exist.

The trickiest guard is the one that catches total failure: if the detective cannot read the payloads at all (the store_nodes call returns nothing), the system quietly lets the meaning detective work alone and logs a warning instead of crashing the whole search. Without this detective, a beginner typing a specific error like “bm25s errors if top_k > corpus size” would get zero matches—the meaning detective only finds similar concepts, never the exact text that matters.

BM25 leg hydration from Qdrant payloads, with top-k clamping and dense-only fallback.

retrievers = [index.as_retriever(similarity_top_k=retrieve_k)]
nodes = store_nodes(index)   # hydrate corpus text from Qdrant payloads
if nodes:
    retrievers.append(BM25Retriever.from_defaults(
        nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))))
else:
    # dense-only fusion fallback
    retrievers = [index.as_retriever(similarity_top_k=retrieve_k)]

The subsystem’s ordered mechanism begins when the hybrid engine is built: the service calls store_nodes(index) to hydrate node texts from Qdrant point payloads — the same store that holds the dense vectors — and then constructs an in-process BM25Retriever via BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))). The similarity_top_k is clamped to the corpus size because the bm25s backend errors when top_k exceeds the number of documents. After both legs retrieve their top-20, they are fused by LlamaIndex’s QueryFusionRetriever in reciprocal_rerank mode with num_queries=1 — a plain two-leg fusion without LLM query generation. The fused candidates then pass through node postprocessors in a deliberate order: injection fencing first, the cross-encoder reranker second. On failure — for example, if hydration returns no nodes or the BM25 package is unavailable — the engine degrades to dense-only fusion with a log line rather than failing the query path.

The design preserves a single-source-of-truth invariant: the corpus text is always derived from the same Qdrant point payloads that hold the dense vectors, so there is no second index to drift out of sync. Re-ingesting the corpus updates both legs atomically because the sparse leg is rebuilt from the same payloads. This invariant guarantees that the text used for BM25 scoring is always consistent with the text stored alongside the dense vectors, eliminating a class of silent data corruption that would arise if a separate index drifted due to partial updates or misaligned tokenization.

The key trade-off is simplicity against memory cost. By hydrating the corpus in-process, the architecture avoids maintaining a separate, independently ingested sparse index — the obvious alternative being Qdrant native sparse vectors, which would require a second representation (sparse vectors) to be stored and kept consistent with the dense payloads. The cost avoided is the operational burden of synchronizing two indices, including the risk of drift when only one is updated. The price is that the entire corpus (~22,000 nodes) must reside in process RAM for the BM25 leg to operate. On the memory-capped production host (a 512 Mi free-tier container), this price is too high, so HYBRID is switched off, and the sparse leg is shed — queries that would rely on exact identifiers, API names, or error strings lose coverage.

One concrete failure mode is analyzer mismatch between index and query time. This occurs when the tokenizer used at hydration (implicitly, the one built into bm25s via the default analyzer) differs from the tokenizer applied to the query string, causing silent recall loss on stemmed or split terms. An operator would see the signal by evaluating recall separately for each leg: if the dense leg retrieves a relevant document but the BM25 leg does not, and the query contains an exact compound term (e.g., mode="reciprocal_rerank"), the operator should inspect the term-level matches. The symptom is low overlap between the legs’ top-20 results — high overlap would indicate redundancy, low overlap coverage, but a persistent failure of the sparse leg to catch exact terms points directly to an analyzer mismatch, measurable as a drop in recall@k for those queries relative to a controlled BM25 baseline.

Empty nodes from hydration

• Trigger – store_nodes(index) returns an empty list nodes because the Qdrant store has no points or the payloads lack text.
• Guard – The if nodes: condition that guards the retrievers.append(BM25Retriever.from_defaults(...)) call; when nodes is falsy, the sparse leg is skipped.
• Posture – fail-soft: the retrieval degrades to dense-only fusion, continuing the query path.
• Operator signal – The exact log line: print("[hybrid] no nodes in store — dense-only fusion").
• Recovery – No retry or fallback value; the operator must manually investigate why the Qdrant store contains no points or why store_nodes(index) returned nothing, then re-run the hybrid engine after populating the store.

Stale BM25 statistics after corpus update

• Trigger – The corpus in Qdrant is re-indexed (nodes added, removed, or changed) without rebuilding the in-process BM25 leg; the term‑frequency and document‑length statistics (IDF, avgdl) baked at the original index time become outdated.
• Guard – Not explicitly provided in source; the text only states that “after large corpus changes the sparse index must be rebuilt, and an in-process BM25 leg must be re-hydrated.” There is no automated guard.
• Posture – fail-soft: the BM25 leg continues to serve queries with stale statistics, causing silent recall loss and score drift.
• Operator signal – No explicit log or error; the degradation is observable only by comparing retrieval quality over time or by noticing that the BM25 leg no longer matches the current corpus composition.
• Recovery – Manual step: rebuild the hybrid engine by re‑running the code that calls store_nodes(index) and creates a new BM25Retriever, so that the in-process leg is re-hydrated from the current Qdrant payloads.

Top‑k clamp triggered by small corpus

• Trigger – The configured retrieve_k value is larger than len(nodes) (e.g., during testing with a tiny index).
• Guard – min(retrieve_k, len(nodes)) used as the similarity_top_k argument in BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))).
• Posture – fail-soft: the BM25 leg silently returns at most len(nodes) results instead of the requested retrieve_k.
• Operator signal – No log line; the operator may observe that the number of results from the BM25 leg is lower than expected, or see fewer than retrieve_k items in the fusion output.
• Recovery – No automated recovery; the operator can adjust retrieve_k or expand the corpus.

BM25 package unavailable

• Trigger – The bm25s package (or its dependencies) is missing from the runtime environment, causing an import or instantiation failure of BM25Retriever.
• Guard – Not explicitly shown in source; the text states that “if the BM25 package is unavailable, the engine degrades to dense-only fusion with a log line,” but the code snippet does not include a try‑except or import guard. The guard is absent from the provided context.
• Posture – fail-hard if no guard exists (the query path would crash), but the description implies a soft fallback; since no exact guard identifier is given, the posture is ambiguous.
• Operator signal – If the guard is missing, a traceback from ImportError or from BM25Retriever.from_defaults would appear. If a fallback is implemented, the log line "[hybrid] no nodes in store — dense-only fusion" (or a similar fallback message) would be printed, but the source does not specify a distinct signal for this case.
• Recovery – Manual: install the bm25s package and restart the service, or rely on the (unspecified) fallback to dense-only fusion.

store_nodes(index) raises an exception

• Trigger – The Qdrant connection is unavailable, the index object is invalid, or the store_nodes function itself throws an error (e.g., due to payload parsing failure).
• Guard – Not provided in source; the call nodes = store_nodes(index) is unprotected and any exception propagates.
• Posture – fail-hard: the query path aborts with an unhandled exception before the hybrid retriever is built.
• Operator signal – An uncaught traceback from store_nodes(index), typically referencing Qdrant client errors or attribute errors.
• Recovery – Manual: diagnose the Qdrant connection or the index object state, fix the underlying issue, and restart the query flow.

BM25Retriever instantiation fails due to bad node content

• Trigger – nodes is a non‑empty list, but one or more nodes contain None or malformed text that BM25Retriever.from_defaults cannot process (e.g., missing node text in the payload).
• Guard – Not shown in source; no validation of node text before passing to BM25Retriever.from_defaults.
• Posture – fail-hard: the from_defaults call raises an exception, crashing the hybrid engine build.
• Operator signal – A traceback from BM25Retriever.from_defaults indicating a TypeError or ValueError related to node text.
• Recovery – Manual: inspect the Qdrant payloads of the ingested nodes, ensure each node has a valid text field, and re‑run the hybrid engine or reprocess the corpus.

Making a BM25 retriever trustworthy starts with how you split text into tokens—like a chef deciding what counts as a single ingredient when reading a recipe. This system is for making sure the search engine actually finds the right documents when you ask for something specific, especially in technical code or documentation.

Now walk through what actually happens. The chef first decides whether “chop” and “chopped” are the same ingredient (stemming)—a light stemmer like Porter helps, but an aggressive one might merge distinct identifiers like “retriever” and “retrieval” into one, losing precision. For technical text, a compound term like “QueryFusionRetriever” can be split on camelCase into three tokens (query, fusion, retriever), but splitting loses the exact-match advantage you want for rare names. The biggest bug hides here: if the recipe index (index time) has one set of splitting rules and the query (query time) uses different rules—say the index stemmed but the query didn’t—then a search for “retrievers” finds nothing, because the two sides don’t speak the same language. The numeric parameters (k_1) and (b) also matter: (k_1) controls how much repeating a word counts (low for short titles, high for long documents), while (b) adjusts for document length—but if your documents are already all the same chunk size, length normalization does little.

Without this careful tokenization, a beginner typing an exact function name like similarity_top_k would get zero results because the index split it into parts while the query kept it whole, or vice versa. The search silently fails on the very terms that should match—and you’d never know why.

Hybrid retrieval construction with BM25 clamping and fallback ensures robust sparse leg integration.

retrievers = [index.as_retriever(similarity_top_k=retrieve_k)]   # dense leg
nodes = store_nodes(index)   # hydrate corpus text back out of Qdrant payloads
if nodes:
    # bm25s errors if top_k > corpus size — clamp for small indices.
    retrievers.append(BM25Retriever.from_defaults(
        nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))))
else:
    print("[hybrid] no nodes in store — dense-only fusion")

The subsystem begins when the hybrid engine is built: the dense leg is instantiated first via index.as_retriever(similarity_top_k=retrieve_k), retrieving the top-20 documents from Qdrant. Next, the sparse leg is constructed by hydrating all node texts from the Qdrant payloads using store_nodes(index), which returns the corpus as a list of nodes. These nodes are passed to BM25Retriever.from_defaults(nodes=nodes, similarity_top_k=min(retrieve_k, len(nodes))). The min guard is a defensive clamp because the bm25s backend errors if top_k exceeds the corpus size—this ensures the engine can be built even for small test indices. If hydration fails (returns no nodes or the BM25 package is unavailable), the engine degrades to dense-only fusion with a log line rather than failing the query path. Both legs retrieve their top-20 lists, which are then fused by QueryFusionRetriever in reciprocal_rerank mode with num_queries=1. The fused candidates then pass through node postprocessors in deliberate order: injection fencing first, then the cross-encoder reranker second. When the cross-encoder is enabled, the fusion cut is widened to the reranker's candidate budget so the reranker has real candidates to demote before truncating back to the final top-k.

The design preserves the invariant of single source of truth: the corpus lives in one Qdrant store as dense vectors with node text and metadata in the payload. The BM25 leg does not get its own separately-ingested index. Instead, it reconstructs the sparse representation on demand by hydrating the same payloads. This ensures that re-ingesting the corpus updates both legs simultaneously—there is no second index to drift out of sync. The guarantee is that at the moment the engine is built, the sparse leg operates on exactly the same textual corpus as the dense leg, preventing silent divergence between the two representations.

The key trade-off is between in-process simplicity and memory cost. This architecture rejects the obvious alternative of pushing the sparse leg server-side as Qdrant native sparse vectors, which would require maintaining a second representation in the collection. That alternative incurs the cost of extra pipeline steps for indexing and updating a separate sparse vector model, plus the operational burden of keeping two representations consistent. Instead, the in-process BM25 leg buys single-source-of-truth simplicity at the price of holding the full corpus text in process memory (22k hydrated nodes). On a memory-capped production host (512 MiB container), this cost is untenable, and HYBRID is switched off to run only the dense leg. The rejection of the server-side alternative avoids the ongoing maintenance cost of a second representation, but it limits deployment to hosts that can fit the corpus in RAM.

One concrete failure mode is an analyzer mismatch between index and query time—for example, if the tokenizer used during Qdrant ingestion applies a stemmer like "retrievers" → "retriever" but the BM25Retriever at query time uses a different analyzer that does not stem, or splits identifiers differently. This causes silent recall loss on the very terms that should match (e.g., exact package names or error strings). The operator would see no error messages or warnings, but the overlap statistics between the two legs (how many of the top-20 are shared) would suddenly drop, indicating that the sparse leg is failing to catch exact identifiers that used to appear. The signal is a sudden decrease in recall@k on queries containing those identifiers, detectable by comparing the leg's performance before and after any analyzer configuration change. The design's reliance on in-process hydration from Qdrant payloads also means that if the payload text itself was generated with a different analyzer than the one used at build time, the mismatch is baked in at the point of store_nodes.

Analyzer mismatch between index and query time

• Trigger – Mismatching analyzers at index and query time (e.g., different stemmer or tokenizer for the same field).
• Guard – No guard shown in source.
• Posture – fail-soft: retrieval silently degrades with no error or exception.
• Operator signal – “silent recall loss on stemmed/split terms” – no log line; recall metrics would drop.
• Recovery – Manual consistency check: ensure the same analyzer is used for both indexing and querying.

Stale corpus statistics

• Trigger – Large corpus changes after index time without rebuilding the sparse index (IDF and avgdl become outdated).
• Guard – No guard shown in source; the source states “the sparse index must be rebuilt, and an in-process BM25 leg must be re-hydrated.”
• Posture – fail-soft: retrieval continues with stale statistics, impacting BM25 scoring quality.
• Operator signal – No explicit signal; scoring drifts silently.
• Recovery – Manually rebuild the sparse index and re-hydrate the BM25 leg.

Fusion dilution

• Trigger – A weak sparse leg (tiny corpus, bad analyzer) injects noise into a strong dense ranking during RRF fusion.
• Guard – No guard shown in source; listed as a warning.
• Posture – fail-soft: hybrid underperforms the best standalone leg (as noted: “measured pipelines have seen hybrid underperform the best standalone leg”).
• Operator signal – No direct alert; observable only via metric comparison (hybrid < best single leg).
• Recovery – Improve the sparse leg (larger corpus, better analyzer) or adjust fusion weights manually.

Score misuse

• Trigger – Treating BM25 scores as probabilities or applying global thresholds across queries or corpora.
• Guard – No guard shown in source; the source warns “fuse by rank, not by raw score.”
• Posture – fail-soft: incorrect retrieval decisions are made without an error.
• Operator signal – No explicit signal; results may be degraded with no alert.
• Recovery – Use rank-based fusion (RRF) instead of thresholding; re-architect scoring pipelines.

bm25s top_k > corpus size

• Trigger – Asking for more results than the corpus contains (e.g., similarity_top_k > len(nodes)).
• Guard – Explicit clamp: min(retrieve_k, len(nodes)) passed as similarity_top_k in BM25Retriever.from_defaults().
• Posture – fail-soft: similarity_top_k is silently reduced; no exception raised.
• Operator signal – No log line; silent clamping.
• Recovery – Automatic: the minimum value is used, preventing the error.

Hydration failure (no nodes or BM25 package unavailable)

• Trigger – store_nodes(index) returns empty (e.g., Qdrant payload missing) or the BM25 package is not installed.
• Guard – Conditional: if nodes: only then BM25 leg is added; else print("[hybrid] no nodes in store — dense-only fusion").
• Posture – fail-soft: degrades to dense-only retrieval; no query failure.
• Operator signal – Printed log line: "[hybrid] no nodes in store — dense-only fusion".
• Recovery – Automatically continues with only the dense leg; no retry.