microsoft / microsoft/foundry-dev-tools

Add per-model `stream: false` option in my-models.yml for non-streaming custom endpoints

Open
#353 2 comments 1 reaction 1 assignee View on GitHub

@MuyangAmigo is already working on this.

Since Feb 26, 2026.

feature request needs attention
Dominant language
JavaScript
Stars
2.1k
Forks
260
Avg merge
42m
Merged PRs (30d)
29

Description

Feature Request

Summary

Add a stream configuration option per model in my-models.yml so users can set stream: false for custom endpoints that do not support Server-Sent Events (SSE) streaming.

Current Behavior

AI Toolkit always sends "stream": true when calling custom model endpoints. If the endpoint does not support streaming, the request fails with a 500 error, making the model completely unusable in AI Toolkit even if the endpoint works correctly for non-streaming requests.

Expected Behavior

Allow users to opt out of streaming per model in my-models.yml:

- name: Claude Opus 4.6 v1
  chat_completion_url: https://my-gateway.example.com/v1/chat/completions
  stream: false   # <-- new option
  api_model_name: my-model # e.g. claude-opus-4.6

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.