docling-project / docling-project/docling-parse
Bound the native content-stream parse on vector-dense pages (path-storm PDFs cost GiBs per page in `get_page()`)
- Dominant language
- C++
- Stars
- 333
- Forks
- 80
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 10
Description
## Summary
`DoclingPdfParser.get_page()` parses and materialises every path in a page's content stream before any `ContentConfig` level or `create_words`/`create_textlines` flag can bound the work. On CAD / wiring-schematic pages drawn as hundreds of thousands of vector paths this costs multiple GiB of native memory per page, and there is no configuration that limits it. Downstream this is the OOM class tracked in docling-project/docling#4058.
## Measurements
Synthetic path-storm page (dense ruled grid of N line segments, generator in docling#4058), Docling 2.121 / docling-parse 7.14.0, amd64, 4 CPU:
| segments | peak RSS |
|---|---|
| 100k | ~0.6 GiB |
| 1M | 3.49 GiB (default backend, tables on or off identical) |
| 1M, all `ContentLevel` SKIP + `compute_shapes=False` | 2.75 GiB — the parse itself dominates, not materialisation |
| 1M, pypdfium2 backend | 1.42 GiB, 3× faster |
Real documents (equipment manuals, 337–480 pages, worst pages 466k–970k paths, 12 GiB cgroup, 40-page batches, Docling 2.121/2.122 with docling-parse 7.14.0):
| document | docling-parse default | `OcrMode.FULL_PAGE` (decode skipped since docling#4061) | pypdfium2 backend |
|---|---|---|---|
| 352 pp, worst page 970k paths | OOMKilled at 12 GiB | 6.55 GiB peak | 6.16 GiB peak |
| 337 pp, worst page 660k paths | OOMKilled at 12 GiB (2.41) | 6.19 GiB | 6.03 GiB |
| 480 pp, worst page 466k paths | 11.34 GiB peak, no headroom | 7.25 GiB | 4.98 GiB |
Full details and the docling-side discussion: docling-project/docling#4058 (this issue is the docling-parse half that thread left open).
## Ask
A per-page budget or degradation threshold inside `get_page()` — for example a maximum path/segment count (or a byte budget on the decoded content stream) above which the parser stops materialising paths for that page and reports it as degraded, so callers can fall back (OCR-only, or a different backend) without the process being killed first. The docling side is already able to consume a `parsed_page=None`-style outcome since docling#4061.
Happy to contribute the change if maintainers can point at the preferred place for such a budget in the C++ parser.
Contributor guide
Research direction
Start at DoclingPdfParser.get_page() and trace the native C++ content-stream parse and path materialisation. Compare the existing ContentConfig, ContentLevel, and compute_shapes handling, then define a per-page path or byte budget that yields the parsed_page=None-style degraded outcome for docling#4061 to consume. Done means path-storm pages avoid OOM before materialisation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100