deepset-ai / deepset-ai/haystack
LLMDocumentContentExtractor treats a chat generator's own "error" field as a failed call
@julian-risch is already working on this.
Since Sep 13, 2026.
- Dominant language
- Python
- Stars
- 26.6k
- Forks
- 3.2k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 194
Description
LLMDocumentContentExtractor decides whether the per-document LLM call failed by checking if "error" in result on the chat generator's output dict. Generator outputs are free to carry extra fields, and a wrapper-style generator that always includes an error field (for example {"replies": [...], "error": None}) makes every document land in failed_documents with extraction_error=None, even though the call succeeded and returned usable content.
This is the same internal-sentinel collision class as the LLMMetadataExtractor bug fixed in #12685, on the other extractor.
Reproducer:
from unittest.mock import patch
from haystack import Document
from haystack.components.converters.image.document_to_image import DocumentToImageContent
from haystack.components.extractors.image import LLMDocumentContentExtractor
from haystack.dataclasses.chat_message import ChatMessage, ImageContent
class ErrorKeyChatGenerator:
"""Wrapper-style generator whose output dict always includes an "error" field."""
def run(self, messages, **kwargs):
return {"replies": [ChatMessage.from_assistant("extracted text")], "error": None}
extractor = LLMDocumentContentExtractor(chat_generator=ErrorKeyChatGenerator())
with patch.object(DocumentToImageContent, "run") as mock_convert:
mock_convert.return_value = {
"image_contents": [ImageContent.from_file_path("./test/test_files/images/apple.jpg")]
}
doc = Document(content="", meta={"file_path": "/path/to/image.pdf"})
result = extractor.run(documents=[doc])
print({k: len(v) for k, v in result.items()})
Output (run twice, identical both times): {'documents': 0, 'failed_documents': 1} - the document is dropped with extraction_error=None even though the generator returned valid content. Expected: the document is processed and its content becomes "extracted text".
Proposed fix: in _process_llm_results, check "replies" not in result instead of "error" in result. The internal failure dicts never carry a replies key and generator outputs always do, so the failure sentinel no longer shares a namespace with generator output fields. PR follows.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.