Back to Practice

Chunking with SentenceSplitter

Full
rag

Task: Chunking with SentenceSplitter

Write a Python function split_documents_into_chunks(docs: list) that takes a list of Document objects (from llama_index.core) and returns a list of TextNode objects obtained by splitting each document using a SentenceSplitter. The splitter must be configured using the environment variables CHUNK_SIZE (default 512) and CHUNK_OVERLAP (default 64). Use os.environ.get() to read these values.

Inside the function, include a comment (or docstring) that explains:

  • Why embedding an entire file as a single vector is suboptimal (the chunk size limit and loss of granularity).
  • How overlap helps preserve facts that straddle chunk boundaries.

Your implementation must use the exact same identifiers and imports as shown in the repository source (e.g., SentenceSplitter, get_nodes_from_documents).

Your code
Sources
  • roadmap-kg/kg/ground_content.py:218-261
  • roadmap-kg/kg/memory_common.py:757-804
  • roadmap-kg/kg/memory_common.py:683-728