deepset-ai / deepset-ai/haystack

Bug: MarkdownHeaderSplitter shares parent_headers list across secondary chunks when keep_headers=False

Open
#12,691 1 comment 0 reactions 1 assignee View on GitHub

@sjrl is already working on this.

Since Sep 14, 2026.

P2
Dominant language
Python
Stars
26.6k
Forks
3.2k
Avg merge
1d 3h
Merged PRs (30d)
194

Description

Describe the bug
When MarkdownHeaderSplitter runs with keep_headers=False and a secondary_split (e.g. "word"), every secondary chunk produced from the same header section shares the same parent_headers list object. Mutating one chunk's parent_headers leaks into its siblings (same shape as #12248 / #12424, but on a conditional path those PRs did not cover).

Error message
No error; the metadata is silently aliased.

Expected behavior
Each chunk gets its own copy of parent_headers (as already happens with keep_headers=True + secondary split, and as #12424 did for PythonCodeSplitter). Editing one chunk must not affect its siblings.

To Reproduce

from haystack import Document
from haystack.components.preprocessors.markdown_header_splitter import MarkdownHeaderSplitter

text = "# Top\n\n## Section A\n\n" + "word " * 100
sp = MarkdownHeaderSplitter(keep_headers=False, secondary_split="word", split_length=10, split_overlap=0)
sp.warm_up()
out = sp.run(documents=[Document(content=text, meta={"tags": ["orig"]} )])["documents"]
print(len(out))  # >1 from the same Section A parent
print(out[0].meta["parent_headers"] is out[1].meta["parent_headers"])  # True (bug; expect False)
out[0].meta["parent_headers"].append("MUT")
print(out[1].meta["parent_headers"])  # ['Top', 'MUT'] (bug; expect ['Top'])

Verified on main @ 5bf0ffc (v3.2.0-rc0): with keep_headers=True the identity check is False and the mutation does not leak; with keep_headers=False it is True and leaks. tags (deep-copied by the secondary DocumentSplitter) is safe in both modes, so only the header-preservation overwrite leaks.

Additional context
Root cause in haystack/components/preprocessors/markdown_header_splitter.py _apply_secondary_splitting (~L277-279 on origin/main):

if not self.keep_headers:
    for key in ["header", "parent_headers"]:
        if key in doc.meta:
            split.meta[key] = doc.meta[key]  # shares the list object across all secondary siblings

The secondary DocumentSplitter already deep-copies each chunk's meta, but this loop then overwrites parent_headers with the parent doc's list reference, discarding the distinct copies. header is a str (safe); parent_headers is a list (unsafe).

Sibling history (not duplicates): #12248 (family issue) -> #12249 fixed Markdown/CSV/hierarchical; #12424 fixed PythonCodeSplitter and stated the family was closed after checking the remaining splitters. This site was missed because it only triggers when both keep_headers=False and secondary_split are set. Distinct from #12618/#12619 (page_number drift with overlap) and #12477 (leading-header loss).

Possible fixes:
a) (Recommended, minimal, matches #12424 pattern) split.meta[key] = deepcopy(doc.meta[key]) at the overwrite site -- deepcopy is already imported in this file.
b) Copy just the list (list(...)) for parent_headers -- narrower but leaves any deeper nesting shared.
c) Deep-copy clean_meta upfront and keep the overwrite deep-copied -- redundant; (a) alone suffices since the secondary outputs are otherwise already distinct.

FAQ Check

System:

  • OS: macOS
  • Haystack version: main @ 5bf0ffc (VERSION.txt 3.2.0-rc0), Python 3.11

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.