docling-project / docling-project/docling
Actually Using OpenAI for Annotation
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Question
I have seen that it should be possible to interact with the OpenAI image API for image annotation.
e.g. in this PR: https://github.com/docling-project/docling/issues/192
Now, I have been wondering, how excatly this should be done?
It seems that OpenAI expects:
```json
{
"model": "gpt-4-vision-preview",
"messages": [
{
"role": "user",
"content": [
{ "type": "text", "text": "Describe the image in three sentences." },
{ "type": "image_url", "image_url": { "url": "data:image/png;base64,..."} }
]
}
],
"max_tokens": 200
}
```
But PictureDescriptionApiOptions gives:
```json
{
"model": "gpt-4-vision-preview",
"prompt": "...",
"image": ""
}
```
Contributor guide
Assessment
This issue has not been assessed yet.