aws / aws/nova-prompt-optimizer
add support for OCR/Vision Prompting
- Dominant language
- Python
- Stars
- 56
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
To add vision/OCR support, you'd need to modify at minimum:
1. BedrockConverseHandler._get_messages() — Support image content blocks alongside text:
# Bedrock Converse API expects this for images:
{"image": {"format": "png", "source": {"bytes": }}}
2. PromptAdapter — Add a way to specify that a template variable should be populated with image data
rather than text.
3. DatasetAdapter — Handle dataset rows where an input column contains image file paths, loading the
bytes at inference time.
4. InferenceAdapter — Thread image data through to the Bedrock call.
Contributor guide
Research direction
Start by tracing BedrockConverseHandler._get_messages(), PromptAdapter, DatasetAdapter, and InferenceAdapter to understand how text inputs move to the Bedrock call. Define how image paths become bytes and how image data is represented through each adapter, using the shown Converse API image block as the target. Done means dataset image inputs reach Bedrock alongside text for OCR or vision prompting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- ai, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100