docling-project / docling-project/docling
Error in StandardPdfPipeline: PIL.Image.fromarray TypeError for unusual NumPy array shape ((1, 1, 26), '|u1')
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Description
Using Docling as the core of my document-extraction pipeline, I encountered an error while processing a specific PDF file.
The pipeline fails inside the StandardPdfPipeline, specifically during the code-formula enrichment step that relies on the idefics3 image processor.
It seems that the Idefics3 image processor tries to convert an extracted image into a PIL image, but the underlying NumPy array has an unexpected shape, which PIL cannot interpret.
### Error Trace
```
Error processing filename.pdf: Pipeline StandardPdfPipeline failed
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/PIL/Image.py", line 3308, in fromarray
mode, rawmode = _fromarray_typemap[typekey]
KeyError: ((1, 1, 26), '|u1')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/docling/pipeline/base_pipeline.py", line 75, in execute
conv_res = self._enrich_document(conv_res)
...
File "/usr/local/lib/python3.10/dist-packages/transformers/image_transforms.py", line 213, in to_pil_image
return PIL.Image.fromarray(image, mode=image_mode)
File "/usr/local/lib/python3.10/dist-packages/PIL/Image.py", line 3312, in fromarray
raise TypeError(msg) from e
TypeError: Cannot handle this data type: (1, 1, 26), |u1
```
### Steps to reproduce
1. Run Docling with StandardPdfPipeline()
2. Process PDF (I will add the file if needed)
```
from docling.pipeline.standard_pdf_pipeline import StandardPdfPipeline
from docling.backend.pypdfium2_backend import PyPdfiumDocumentBackend
from docling.datamodel.pipeline_options import (
PdfPipelineOptions,
TableFormerMode,
EasyOcrOptions,
)
..
..
..
self.ocr_options = EasyOcrOptions(force_full_page_ocr=False)
self.backend = PyPdfiumDocumentBackend
pipeline_opts = PdfPipelineOptions(
do_code_enrichment = True,
do_formula_enrichment = True,
ocr_options=self.ocr_options,
do_ocr=do_ocr,
generate_page_images=False,
generate_picture_images=True,
images_scale=2.0,
do_table_structure=True,
profile_pipeline_timings=True
)
table_mode: TableFormerMode = TableFormerMode.ACCURATE
pipeline_opts.table_structure_options.mode = table_mode
pipeline_opts.table_structure_options.do_cell_matching = False
format_options = {}
format_options[InputFormat.PDF] = PdfFormatOption(
pipeline_cls=StandardPdfPipeline,
backend= self.backend,
pipeline_options=pipeline_opts
)
```
```
def convert_document(self, file_path: str) -> Any:
"""Convert document in DoclingDocument."""
result = self.converter.convert(source=file_path)
return result.document
```
### Expected Behavior
The PDF should be processed entirely, including formula detection, without raising image-processing errors.
### Actual Behavior
The Idefics3 processor receives an image array with shape (1, 1, 26) and dtype |u1, which causes a failure during conversion to PIL:
- shape has 26 channels (!?)
- PIL does not support this format
- processing stops before completing enrichment
### Environment
Docling version:
```
Docling version: 2.63.0
Docling Core version: 2.52.0
Docling IBM Models version: 3.10.2
Docling Parse version: 4.7.1
Python: cpython-310 (3.10.12)
Platform: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35
```
#### Other dependencies:
- transformers: 4.57.3
- Python 3.10.12
#### OS (Docker container):
Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35
### Additional context
This might be caused by:
- An incorrectly extracted artifact from the PDF (maybe a corrupted image)
- The formula detection processor not handling edge-case images
### A possible fix might involve:
- Validating or sanitizing image arrays before passing them to the Idefics3 processor
- Converting unusual multi-channel images to RGB or grayscale
- Skipping invalid images with a warning
Happy to provide the PDF file if needed.
Contributor guide
Assessment
This issue has not been assessed yet.