Single-store text_key / index_doc_id

Worked Example: Deterministic Node IDs & Collection Naming for Idempotent Qdrant Indexing

This example demonstrates the pattern used in the real build_index_from_nodes function to ensure that re-indexing the same corpus always upserts (overwrites) existing Qdrant points rather than appending duplicates. The key ideas are:

1. Deterministic node IDs – each node gets a uuid5 identifier derived from its text (or content) and an optional namespace. Same text → same ID, so an upsert overwrites the old point.
2. Corpus-fingerprinted collection name – the collection name is built from the sorted set of all node IDs via SHA-256 (first 12 hex digits). Any change to the corpus (new, removed, or altered nodes) yields a new collection, stale chunks never leak into retrieval.
3. Single store (Qdrant as docstore) – when using Qdrant, the in-memory index.docstore is left empty by design. All text and metadata live in the Qdrant point payload (text_key="text", index_doc_id=True). This avoids duplication and makes Qdrant the single source of truth. The helper store_nodes() falls back to scrolling points out of the vector store when the docstore is empty.

Step-by-step walkthrough

Suppose we have a list of TextNode objects from a corpus. We want to index them into Qdrant idempotently. The solution does:

• Assign stable IDs: loop over nodes, set node.node_id = uuid.uuid5(uuid.NAMESPACE_DNS, f"{namespace}|{node.text}") (or similar content-based scheme).
• Compute collection name: ns = re.sub(r"[^a-zA-Z0-9_-]", "_", namespace) or "content" fp = hashlib.sha256('\x1f'.join(sorted(n.node_id for n in nodes)).encode()).hexdigest()[:12] return f"kg-{ns}-{fp}"
• Build index with a QdrantVectorStore configured with index_doc_id=True, text_key="text" and the computed collection. This causes LlamaIndex to skip storing nodes in the docstore.
• Verify empty docstore: after building the index, index.docstore.docs is empty (or only contains placeholder entries). Calling store_nodes(index) retrieves the nodes from Qdrant instead. We print that to confirm the pattern.

Study the code below to see each piece in action.