microsoft / microsoft/foundry-dev-tools
Add per-model `stream: false` option in my-models.yml for non-streaming custom endpoints
@MuyangAmigo is already working on this.
Since Feb 26, 2026.
- Dominant language
- JavaScript
- Stars
- 2.1k
- Forks
- 260
- Avg merge
- 42m
- Merged PRs (30d)
- 29
Description
Feature Request
Summary
Add a stream configuration option per model in my-models.yml so users can set stream: false for custom endpoints that do not support Server-Sent Events (SSE) streaming.
Current Behavior
AI Toolkit always sends "stream": true when calling custom model endpoints. If the endpoint does not support streaming, the request fails with a 500 error, making the model completely unusable in AI Toolkit even if the endpoint works correctly for non-streaming requests.
Expected Behavior
Allow users to opt out of streaming per model in my-models.yml:
- name: Claude Opus 4.6 v1
chat_completion_url: https://my-gateway.example.com/v1/chat/completions
stream: false # <-- new option
api_model_name: my-model # e.g. claude-opus-4.6
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.