Back to the stack guide
RAG

LlamaIndex ingestion & retrieval

IngestionPipeline with content-keyed caching, stable node IDs, LlamaParse for external docs — an idempotent pipeline where nothing recomputes unless content changed.

The pipeline is idempotent end to end

Ingestion runs through LlamaIndex's IngestionPipeline with an IngestionCache keyed on content hash: unchanged nodes are never re-embedded (re-sync: 392s → 4s). Nodes carry stable, content-derived IDs, so upserts into Qdrant replace rather than duplicate, and every downstream cache — embeddings, judge verdicts, explanations — keys cleanly across runs.

The chain is: content hash → stable ID → cached embedding → upsert. Edit one lesson and exactly that lesson's nodes invalidate; nothing else moves.

External documents (PDFs, papers) enter through LlamaParse, gated on an API key and optional by design — the pipeline degrades gracefully without it.

The flagship lane: select-to-explain

Highlight any text on a lesson page and the platform explains it from its own corpus: the selection goes to the RAG service, retrieval runs over the shared store (hybrid in the full profile; dense-only on the memory-capped prod host — see the RAG serving deep dive), and the answer comes back grounded in the retrieved excerpts, with citations.

Grounding is strict: the engine answers only from what it retrieved, and "the excerpts don't cover it" is a legitimate, expected answer. A confident answer with no support isn't style — it's a gate failure.

Two caches on the answer path

  • a persistent explanation cache — repeat queries answer in ~0.05s instead of ~1.9s;
  • a semantic cache in Qdrant — near-duplicate queries on the same page (cosine above threshold) reuse the stored answer.

Between them, most real traffic never touches an LLM.

Retrieval shape

512/64 chunking with a retriever/synthesizer split, so concepts embed clean; one shared corpus with per-namespace views at query time — glossary, tutor cards, and research lanes each see their slice of the same substrate. In the full profile, recall comes from wide hybrid fusion, precision from the cross-encoder reranker, and quality from response evals measuring faithfulness, relevancy, and correctness against pinned retrieval metrics.

Why LlamaIndex

The pipeline leans on primitives that would be expensive to rebuild honestly: IngestionPipeline/IngestionCache for content-keyed idempotence, LlamaParse for messy external documents, StorageContext persistence that snapshots to R2 through fsspec, and the retriever/postprocessor seams the fencing and reranking stages plug into. The framework is load-bearing at the seams, and replaceable inside them.

Numbers

  • Content-keyed re-sync: 392s → 4s; unchanged nodes never re-embed.
  • Repeat explanations: ~0.05s from cache vs ~1.9s cold — most real traffic never touches an LLM.
  • Chunking: 512/64 with stable, content-derived node IDs.
  • Fusion width in the full profile: top-20 per leg before reranking.