docling-project / docling-project/docling
Docling Serve Takes over 15 minutes to parse a 5 page PDF
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 98
Description
### Bug
No matter if I use CUDA docker images or not, and no matter how I setup the compose file, the docling serve tool takes 15 minutes to process a single pdf. The model card said SmolDocling should process each page at around 35 seconds and I cannot get anywhere close to this performance on either a production grade server or a laptop with a 4090 in it.
The docling python installation and use works, but docling serve does not.
I make the api request to docling serve and poll for response and it always times out (currently the max timeout is set to 2400 seconds). This happens with either the async or sync process.
When I poll async all I see in the logs is the initial HTTP request and then the status endpoint being checked by my code, no actual page level or other processes being shown. Also no workers are being spawned.
...
### Steps to reproduce
pull the ghcr.io cu 128:latest image. Set it to a longer max timeout, add the correct artifacts path, exec into it and pre-download the models. Then try and call the vlm local pipeline from the api endpoints.
...
### Docling version
cu128 latest ghcr image
...
### Python version
3.12
...
docker compose file:
version: '3.8'
services:
docling-serve:
image: ghcr.io/docling-project/docling-serve-cu128:latest
container_name: docling-serve
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
limits:
memory: 16G
cpus: '8'
restart: unless-stopped
ports:
- "5001:5001"
# MOUNT THE SAVED CACHE
volumes:
- ./docling-data/cache:/opt/app-root/src/.cache
environment:
# GPU settings
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
# UI and timing
- DOCLING_SERVE_ENABLE_UI=true
- DOCLING_SERVE_MAX_SYNC_WAIT=2400
# Model paths - point to where models actually are
- DOCLING_SERVE_ARTIFACTS_PATH=/opt/app-root/src/.cache/docling/models
- TRANSFORMERS_CACHE=/opt/app-root/src/.cache/huggingface
- HF_HOME=/opt/app-root/src/.cache/huggingface
# GPU and performance
- DOCLING_USE_GPU=true
- OMP_NUM_THREADS=8
- MKL_NUM_THREADS=8
shm_size: '2gb'
logs:
INFO: IP:port - "POST /v1/convert/file/async HTTP/1.1" 200 OK
INFO: IP:Port - "GET /v1/status/poll/8ed9c545-6003-4e8d-8561-aaa219a9445f HTTP/1.1" 200 OK
this status/poll repeats until timeout
docker exec docling-serve ps aux | head -3
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
default 1 139 29.7 60485776 4822956 ? Ssl 18:09 86:15 /opt/app-root/bin/python3 /opt/app-root/bin/docling-serve run
default 112 0.1 3.9 47231076 644204 ? Sl 18:15 0:03 /opt/app-root/bin/python3 /opt/app-root/lib64/python3.12/site-packages/torch/_inductor/compile_worker/__main__.py --pickler=torch._inductor.compile_worker.subproc_pool.SubprocPickler --kind=fork --workers=32 --parent=1 --read-fd=28 --write-fd=32 --torch-key=H4QweOk00IXhifNSBFt33pU0hqEJ8U7VJTQvH3venIU=
docker exec docling-serve ps aux | grep -i "jobkit\|worker" | grep -v "torch" -> This is null
Contributor guide
Research direction
Start by reproducing the issue with the documented docker-compose configuration and the docling-serve run entry point, using /v1/convert/file/async and /v1/status/poll as the observed API flow. Compare the container logs and process list with the working Python installation; done means sync and async requests process the five-page PDF with visible progress and near the reported per-page performance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose, python, pytorch
- Domain
- api, backend, devops, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100