Skip to content

Chunks and provenance

中文

Chunk keeps retrieval content and section context as separate fields:

Field Meaning
body Rendered chunk body. External headings are not rendered here. Headings for internal merged sections remain part of the body.
ancestor_headings Heading path above the chunk's own section.
own_heading The chunk's own (level, title) pair, or None when a chunk represents multiple merged sibling sections.
token_count Authoritative final count: headings, their separator, and body after finalization.
estimated_token_count The split-time estimate. It may differ slightly when incremental planning was used.
headings_token_count / body_token_count The two components of the final count.
start_line / end_line One-based source positions when the parser can determine them.
document_title / document_path Source document identity and provenance.

The heading path is canonical Markdown rendering. When heading_sensitive=True, the splitter includes the rendered heading path and the blank separator before the body in the budget. The metadata is returned regardless of that setting.

SplitResult.document retains document-level metadata and Markdown reference definitions. Use this data to link an indexed chunk back to its source, but do not assume line ranges exist for binary inputs such as DOCX.