docling-project / docling-project/docling
docling-serve OOM during do_chart_extraction = True on 16gb T4 because of default images_scaling = 2
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
docling-serve running OOM on GPU T4 (16gb VRAM) when running do_chart_extraction = True (and do_picture_classification = True) on an image / pdf page where the underlying chart takes up a very large propotion of the page/image. Docling is attempting to get allocated an additional 4gb of VRAM during the enrichment step causing OOM. This does **not** happen when there are multiple charts that take up a large, only when there is 1 'disproportionally large' chart. The reasons seems to be that the chart that is being processed is resized by 2x because of default setting images_scale = 2 (i.e. area is 4x bigger). The granite-vision model's generate() method (called at line 204 of granite_vision.py) seems to need to allocate GPU memory proportional to the number of 384x384 image tiles, and large chart regions at images_scale=2.0 produce too many tiles (9+) to fit in the remaining ~3 GB of T4 VRAM after the 11.4 GB model is loaded.
### Steps to reproduce
Pass a image where the chart takes up a _large_ area into docling-serve with do_chart_extraction = True and do_picture_classification = True. Image with extracted chart sized 668x367 was processed with images_scaling = 1 but not with images_scaling = 2.
Code:
Bash(DOCLING_URL="https://{url}.eastus.azurecontainerapps.io"
RESULT=$(curl -s -X POST "$DOCLING_URL/v1/convert/file/async" \
-F "files=@{filename}.pdf" \
-F "to_formats=md" \
-F "do_chart_extraction=true" \
-F "do_picture_classification=true" \
-F "include_images=false" 2>&1)
However, the bug does not occur when scaling_images = 1.0 (vs. if let unset defaults to 2.0).
### Docling version
docling-serve==1.15.1
### Python version
python 3.11
Contributor guide
Assessment
This issue has not been assessed yet.