docling-project / docling-project/docling
Unable to convert "large" PDFs, silent failures or ERROR docling.pipeline.standard_pdf_pipeline: Stage preprocess failed for run 1, pages [80]: std::bad_alloc
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
I have an 80Mb PDF, which has 184 pages of scanned grayscale images.
Using:
docling --device cuda --to html_split_page --image-export-mode embedded Manual.pdf
Will result in a steady number of `std::bad_alloc` errors.
1. Using `--pdf-backend pypdfium2` avoids this problem completely.
2. Using `--pdf-backend dlparse_v4` similar error as default docling_parse
3. Using `--pdf-backend threaded_docling_parse` silently fails with no output to disk
I get the impression from:
* https://github.com/docling-project/docling/issues/3345
* https://github.com/docling-project/docling-parse/issues/227
That this maybe something under investigation but both issues are closed.
I think the pypdfium2 workaround works OK but comments in https://github.com/docling-project/docling-parse/issues/227#issuecomment-4652392034 suggest there maybe differences in quality depending on the backend
Questions:
1. does changing the default make sense?
2. would supporting batch mode for pages work? I tried `--release-native-memory-every-n-pages 20` with no success
3. would start/end pages support help here as an alternative workaround?
### Steps to reproduce
See bug description
### Docling version
Docling version: 2.104.0
Docling Core version: 2.83.1
Docling IBM Models version: 3.13.3
Docling Parse version: 6.2.0
Python: cpython-312 (3.12.1)
Platform: Windows-11-10.0.22631-SP0
### Python version
Python 3.12.1
This seems redundant, see Docling version above
Contributor guide
Research direction
Start by reproducing the 184-page conversion with the documented docling command and compare the default, pypdfium2, dlparse_v4, and threaded_docling_parse backends. Investigate the reported std::bad_alloc on page 80 and the silent no-output behavior, including whether page batching or start/end page options changes the result. Done means large PDFs complete reliably or fail with a visible, actionable error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100