docling-project / docling-project/docling

per page processing for better speed

Open
#3,960 5 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Requested feature

Currently doclings process document pipeline E2E iterates all pages. Some threading can be configured.

When inferencing models via i.e. vllm and having a machine with 8 gpus and initially 8 documents - things are fine. But some documents are longer than others and this means eventually 7 gpus will be idle - and the 1 remainig (data skew) cannot fully utilize the others.

I would love to find a way forward how to handle data skew in docling in a resource efficient way.

Obviously this is a bit more complicated: Some VLLM hosted models behave differently ad scale to more gpus. I think the question is also how to harmonize this scaling behaviour in a generic way.

Contributor guide

Open the contributing guide

Research direction

No file, test, or concrete entry point is named. Start by tracing the document pipeline E2E and its threading configuration, then reproduce the uneven page-length workload with vLLM across eight GPUs. Done requires an agreed generic approach for per-page processing and data-skew handling, with evidence that resource utilization improves.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.