microsoft / microsoft/amplifier
reasoning_effort is implemented by 5+ providers but absent from core's PROVIDER_CONTRACT.md
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.1k
- Forks
- 261
- Avg merge
- 3h 28m
- Merged PRs (30d)
- 13
Description
Note: This issue is filed on microsoft/amplifier because microsoft/amplifier-core has Issues disabled. It concerns the contract gap in amplifier-core's docs/contracts/PROVIDER_CONTRACT.md.
ChatRequest.reasoning_effort (amplifier_core/message_models.py:212) is a plain str | None — no Literal, no enum, no documented vocabulary. docs/contracts/PROVIDER_CONTRACT.md contains zero mentions of it (verified by grep). The only contract-level guidance is the generic class docstring: "Providers that don't support a field ignore it."
Meanwhile at least five provider modules implement it, with four mutually incompatible vocabularies and different failure behaviors. This gap has now produced a family of real, shipped bugs.
Bugs already traced to this gap
| Fix | Repo | What it was |
|---|---|---|
| #87 (merged) | provider-anthropic | A warning fired for an effort value that was never applied to anything |
| #88 (merged) | provider-anthropic | extended_thinking=False did not suppress output_config.effort; background chores silently ran at max reasoning effort on the primary model |
| #60 (open) | provider-openai | Explicit request.reasoning_effort was not capability-gated → hard InvalidRequestError from the live API on a non-reasoning model |
Each was discovered independently. All three trace to the same root: there is no agreed contract for what this field means, what values are legal, what "off" is, or what a provider must do with a value it cannot honor.
Observed divergence
Directly verified:
| Provider | Accepted values | Ambient config fallback | Notes |
|---|---|---|---|
| anthropic | low/medium/high/xhigh/max (per-model) |
yes | falls back to instance config when request field unset |
| openai | none/low/medium/high/xhigh |
yes (capability-gated) | none is a real "off" token |
| gemini | low/medium/high only |
no | unknown values silently ignored; no off-token in this vocabulary (thinking_budget=0 is the actual off switch) |
| vllm | openai-like | partial | explicit request path not capability-gated (same shape as #60) |
Reported by investigation but not independently re-verified — flagged as such:
| Provider | Reported behavior |
|---|---|
| azure-openai | field entirely unread |
| ollama | forwarded to native think; capability-gated on both paths |
| github-copilot | vocabulary negotiated from the model; gated on both paths; raises on malformed input |
| chat-completions | field entirely unread |
Why this is a contract question, not a provider bug
Fixing each provider individually (as the three PRs above did) does not stop the next occurrence. Concretely undecided today:
- Canonical vocabulary — is
xhigh/maxpart of the portable contract, or provider-specific? - The "off" value — is there a portable way to say "do not reason on this call"? Today
noneworks on openai/copilot, is meaningless on gemini, and is unrecognized on anthropic. There is currently no portable opt-out, which is why callers reach for the non-portableextended_thinking=Falsekwarg — itself undocumented at contract level. - Unsupported-value behavior — ignore, warn, drop, or raise? All four currently occur.
- Ambient-config precedence — may a provider apply its own instance-level default when the request field is unset, even when the request overrides
modelto something else? That specific interaction produced #88.
Suggested outcome
Specify reasoning_effort in PROVIDER_CONTRACT.md: the portable value set, portable "off" semantics, required behavior when a model cannot honor a value, and whether ambient provider config may supply a default. Providers can then be conformed to it — and the next bug in this family becomes a contract violation rather than a surprise.
Filed as an issue rather than a PR because choosing the canonical vocabulary and unsupported-value policy is a maintainer design decision, not something to assert unilaterally across eight provider repos.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with docs/contracts/PROVIDER_CONTRACT.md and the ChatRequest definition at amplifier_core/message_models.py:212, then compare the provider behaviors and linked fixes described in the issue. Done means maintainers have decided and documented the portable vocabulary, off semantics, unsupported-value behavior, and ambient-config precedence so providers can conform to the contract.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100