docling-project / docling-project/docling

feature(picture-description): inject document context (caption, surrounding text, classification) into picture description prompt

Open
#3,498 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Requested feature
Image description backends send every image with the same static prompt, blind to its
context within the document.

We can leverage the surrounding text, caption, or classification label to ground the
description. I will work on this and send a PR, it should be a quick win since all
the information is already available in the `DoclingDocument` at enrichment time.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.