microsoft / microsoft/simplechat

Search summaries, block-revision assist and workflow runners always use the classic GPT endpoint, which V2 cannot configure

Open
#1,427 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
152
Forks
116
Avg merge
7h 7m
Merged PRs (30d)
122

Description

## Summary

Four subsystems read the classic single-endpoint GPT configuration **unconditionally**, with no multi-endpoint branch. They therefore always use `azure_openai_gpt_*` / `enable_gpt_apim` / `azure_apim_gpt_*`, regardless of whether `enable_multi_model_endpoints` is on.

Because the V2 admin surface deliberately does not carry the classic chat form, **these four subsystems cannot be configured from V2 at all.**

## Which consumers have a connections branch, and which do not

Counting references on `paullizer-react-v2-ui`:

| File | `enable_gpt_apim` | `enable_multi_model_endpoints` | |
|---|---|---|---|
| `functions_search_service.py` | 1 | **0** | no connections branch |
| `functions_block_revision_assist.py` | 1 | **0** | no connections branch |
| `functions_workflow_runner.py` | 1 | **0** | no connections branch |
| `functions_personal_workflows.py` | 1 | **0** | no connections branch |
| `route_backend_chats.py` | 8 | 5 | resolves connections first |
| `functions_documents.py` | 5 | 1 | resolves connections first |
| `route_backend_agents.py` | 4 | 4 | resolves connections first |
| `route_backend_conversation_export.py` | 3 | 1 | resolves connections first |

`_create_summary_client` in `functions_search_service.py` is representative — it builds its client straight from the classic settings with no connections lookup anywhere in the function.

## Why this matters

The AI Models rebuild (#1415, #1419, #1426) made V2's AI Models group **look** complete: connections, chat, embeddings and image generation all have real controls. But an administrator working only in V2 has no way to configure the endpoint that search summarisation, block-revision assist, the workflow runner and personal workflows actually use.

Worse, the failure is quiet. Those subsystems do not fall back to connections — they simply use whatever the classic config holds, which on a deployment that has only ever been configured through V2 may be empty or stale. Nothing in V2 indicates this.

This was partly obscured by wording: it is natural to describe `enable_gpt_apim` as "routes chat through API Management". It does considerably more than that, and the flag's blast radius is wider than the Chat section that now hosts its toggle.

## Suggested directions

Two coherent options, plus a stopgap:

1. **Give these four call sites a connections branch**, so they resolve a model the same way chat, documents, agents and export already do. This is the direction that lets V1 be retired.
2. **Bring the classic single-endpoint chat form into V2** so at least the configuration is reachable, accepting the duplication.
3. **Stopgap:** state it in the docs and in the Chat section's notice — that APIM and the classic endpoint apply to more than chat, and that these subsystems always use them. #1419 has already added a sentence to `docs/admin/ai-models.md` along these lines.

Option 1 is the real fix. Until one of these happens, **V1 cannot be retired**, because there is configuration that exists only there.

## Provenance

Found during the three-phase AI Models rebuild. The #1419 session traced `enable_gpt_apim`'s consumers before writing its help text and correctly declined the narrower "routes chat through API Management" wording; that trace is what surfaced this. Verified independently against `origin/paullizer-react-v2-ui`.

## Related

- #1415 — global model connections (phase 1, merged)
- #1419 — Chat section (phase 2), which declares `enable_gpt_apim`
- #1426 — Embeddings and Image generation (phase 3)
- #1420 — a separate capability-scan defect found in the same work

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with _create_summary_client in functions_search_service.py, then compare its classic-settings lookup with the connections resolution in route_backend_chats.py and functions_documents.py. Trace the corresponding paths in functions_block_revision_assist.py, functions_workflow_runner.py, and functions_personal_workflows.py. Done means all four consumers can use the model configured through V2 multi-model endpoints rather than unconditionally using the classic GPT settings.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.