docling-project / docling-project/docling

Error in StandardPdfPipeline: PIL.Image.fromarray TypeError for unusual NumPy array shape ((1, 1, 26), '|u1')

Open
#2,714 5 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Description

Using Docling as the core of my document-extraction pipeline, I encountered an error while processing a specific PDF file.

The pipeline fails inside the StandardPdfPipeline, specifically during the code-formula enrichment step that relies on the idefics3 image processor.

It seems that the Idefics3 image processor tries to convert an extracted image into a PIL image, but the underlying NumPy array has an unexpected shape, which PIL cannot interpret.

### Error Trace
```
Error processing filename.pdf: Pipeline StandardPdfPipeline failed
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/PIL/Image.py", line 3308, in fromarray
mode, rawmode = _fromarray_typemap[typekey]
KeyError: ((1, 1, 26), '|u1')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/docling/pipeline/base_pipeline.py", line 75, in execute
conv_res = self._enrich_document(conv_res)
...
File "/usr/local/lib/python3.10/dist-packages/transformers/image_transforms.py", line 213, in to_pil_image
return PIL.Image.fromarray(image, mode=image_mode)
File "/usr/local/lib/python3.10/dist-packages/PIL/Image.py", line 3312, in fromarray
raise TypeError(msg) from e
TypeError: Cannot handle this data type: (1, 1, 26), |u1
```

### Steps to reproduce

1. Run Docling with StandardPdfPipeline()
2. Process PDF (I will add the file if needed)

```
from docling.pipeline.standard_pdf_pipeline import StandardPdfPipeline
from docling.backend.pypdfium2_backend import PyPdfiumDocumentBackend
from docling.datamodel.pipeline_options import (
PdfPipelineOptions,
TableFormerMode,
EasyOcrOptions,
)
..
..
..

self.ocr_options = EasyOcrOptions(force_full_page_ocr=False)
self.backend = PyPdfiumDocumentBackend

pipeline_opts = PdfPipelineOptions(
do_code_enrichment = True,
do_formula_enrichment = True,
ocr_options=self.ocr_options,
do_ocr=do_ocr,
generate_page_images=False,
generate_picture_images=True,
images_scale=2.0,
do_table_structure=True,
profile_pipeline_timings=True
)
table_mode: TableFormerMode = TableFormerMode.ACCURATE
pipeline_opts.table_structure_options.mode = table_mode
pipeline_opts.table_structure_options.do_cell_matching = False

format_options = {}
format_options[InputFormat.PDF] = PdfFormatOption(
pipeline_cls=StandardPdfPipeline,
backend= self.backend,
pipeline_options=pipeline_opts
)
```

```
def convert_document(self, file_path: str) -> Any:
"""Convert document in DoclingDocument."""
result = self.converter.convert(source=file_path)
return result.document
```

### Expected Behavior

The PDF should be processed entirely, including formula detection, without raising image-processing errors.

### Actual Behavior

The Idefics3 processor receives an image array with shape (1, 1, 26) and dtype |u1, which causes a failure during conversion to PIL:

- shape has 26 channels (!?)
- PIL does not support this format
- processing stops before completing enrichment

### Environment

Docling version:
```
Docling version: 2.63.0
Docling Core version: 2.52.0
Docling IBM Models version: 3.10.2
Docling Parse version: 4.7.1
Python: cpython-310 (3.10.12)
Platform: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35
```

#### Other dependencies:
- transformers: 4.57.3
- Python 3.10.12

#### OS (Docker container):
Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35

### Additional context

This might be caused by:
- An incorrectly extracted artifact from the PDF (maybe a corrupted image)
- The formula detection processor not handling edge-case images

### A possible fix might involve:
- Validating or sanitizing image arrays before passing them to the Idefics3 processor
- Converting unusual multi-channel images to RGB or grayscale
- Skipping invalid images with a warning

Happy to provide the PDF file if needed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.