docling-project / docling-project/docling
per page processing for better speed
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Requested feature
Currently doclings process document pipeline E2E iterates all pages. Some threading can be configured.
When inferencing models via i.e. vllm and having a machine with 8 gpus and initially 8 documents - things are fine. But some documents are longer than others and this means eventually 7 gpus will be idle - and the 1 remainig (data skew) cannot fully utilize the others.
I would love to find a way forward how to handle data skew in docling in a resource efficient way.
Obviously this is a bit more complicated: Some VLLM hosted models behave differently ad scale to more gpus. I think the question is also how to harmonize this scaling behaviour in a generic way.
Contributor guide
Research direction
No file, test, or concrete entry point is named. Start by tracing the document pipeline E2E and its threading configuration, then reproduce the uneven page-length workload with vLLM across eight GPUs. Done requires an agreed generic approach for per-page processing and data-skew handling, with evidence that resource utilization improves.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100