deepset-ai / deepset-ai/haystack

PythonCodeSplitter with strip_docstrings=True emits invalid Python for docstring-only function/method/class bodies

Open
#12,330 2 comments 0 reactions 1 assignee View on GitHub

@julian-risch is already working on this.

Since Aug 28, 2026.

Dominant language
Python
Stars
26.6k
Forks
3.2k
Avg merge
1d 3h
Merged PRs (30d)
194

Description

Describe the bug

PythonCodeSplitter with strip_docstrings=True emits a syntactically invalid chunk for any function, method, or class whose body consists solely of a docstring, for example a one-line custom exception class or a stub method. The docstring is the unit's only statement, so removing it leaves the def/class header with no body at all.

Error message

SyntaxError: expected an indented block after class definition on line 2

Expected behavior

A unit whose body is only a docstring should either keep the docstring in place or otherwise remain valid Python after strip_docstrings=True. It should never come out unparsable.

To Reproduce

from haystack import Document
from haystack.components.preprocessors import PythonCodeSplitter

source = '''
class MyError(Exception):
    """Custom error raised when the widget explodes."""
'''

splitter = PythonCodeSplitter(min_effective_lines=1, max_effective_lines=10, strip_docstrings=True)
result = splitter.run(documents=[Document(content=source)])

import ast
for chunk in result["documents"]:
    ast.parse(chunk.content)  # raises SyntaxError

Same result for a docstring-only top-level function and a docstring-only method.

Additional context

strip_docstrings=True is documented as a RAG-oriented option to shrink stored chunk content while still letting the docstring influence retrieval via meta_fields_to_embed=["docstrings"]. With it enabled, exactly the compact/simple declarations (one-line custom exceptions, stub/abstract methods) get corrupted into unparsable text before they are embedded or indexed.

System:

  • Haystack version: HEAD (0c4b23dbd)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.