deepset-ai / deepset-ai/haystack
PythonCodeSplitter with strip_docstrings=True emits invalid Python for docstring-only function/method/class bodies
@julian-risch is already working on this.
Since Aug 28, 2026.
- Dominant language
- Python
- Stars
- 26.6k
- Forks
- 3.2k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 194
Description
Describe the bug
PythonCodeSplitter with strip_docstrings=True emits a syntactically invalid chunk for any function, method, or class whose body consists solely of a docstring, for example a one-line custom exception class or a stub method. The docstring is the unit's only statement, so removing it leaves the def/class header with no body at all.
Error message
SyntaxError: expected an indented block after class definition on line 2
Expected behavior
A unit whose body is only a docstring should either keep the docstring in place or otherwise remain valid Python after strip_docstrings=True. It should never come out unparsable.
To Reproduce
from haystack import Document
from haystack.components.preprocessors import PythonCodeSplitter
source = '''
class MyError(Exception):
"""Custom error raised when the widget explodes."""
'''
splitter = PythonCodeSplitter(min_effective_lines=1, max_effective_lines=10, strip_docstrings=True)
result = splitter.run(documents=[Document(content=source)])
import ast
for chunk in result["documents"]:
ast.parse(chunk.content) # raises SyntaxError
Same result for a docstring-only top-level function and a docstring-only method.
Additional context
strip_docstrings=True is documented as a RAG-oriented option to shrink stored chunk content while still letting the docstring influence retrieval via meta_fields_to_embed=["docstrings"]. With it enabled, exactly the compact/simple declarations (one-line custom exceptions, stub/abstract methods) get corrupted into unparsable text before they are embedded or indexed.
System:
- Haystack version: HEAD (
0c4b23dbd)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.