Azure / Azure/azure-rest-api-specs
[Question] Azure OpenAI - Latest API version missing "image" message content for Assistant API
- Dominant language
- TypeSpec
- Stars
- 3.1k
- Forks
- 6k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 444
Description
### API Spec link
https://github.com/Azure/azure-rest-api-specs/tree/main/specification/cognitiveservices/data-plane/AzureOpenAI
### API Spec version
2024-07-01-preview
### Question/Query
## Message Content
When trying to add non-text messages we get an error message indicating only "text" is allowed. OpenAI has supported this for a couple months so it's surprising to see this missing still. Do we have an ETA on when this will be added?
```py
await client.beta.threads.messages.create(
thread_id=thread.id,
role="user",
content=[{'text': 'what does this say?', 'type': 'text'}, {'image_url': {'url': 'data:image/png;base64,iVBORw0KG...', 'detail': 'auto'}, 'type': 'image_url'}]
)
```
```bash
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid message content: only messages with 'type=text' are supported currently.", 'type': 'invalid_request_error', 'param': 'content', 'code': 'invalid_type'}}
```
### Environment
_No response_
Contributor guide
Research direction
Start with the specification/cognitiveservices/data-plane/AzureOpenAI path linked in the issue and inspect the 2024-07-01-preview definition for Assistant API message content. Compare its supported content types with the reported Python request and error; done would require a documented, accepted image message shape and confirmation that the Azure endpoint supports it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, openapi
- Domain
- api, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100