github / github/copilot-cli

Reasoning effort "xhigh" silently downgraded to "medium" (not "max") on models without xhigh (e.g. claude-opus-4.6, claude-sonnet-4.6)

Aberta
#3,823 0 comentários 0 reações 0 responsáveis Ver no GitHub
area:models
Linguagem predominante
Shell
Estrelas
11.2k
Forks
1.9k
Merge médio
14h 16min
PRs com merge (30d)
6

Descrição

## Summary

When a reasoning effort of `xhigh` is configured (persisted `effortLevel` in settings, or selected via `/model`) and the active model does **not** advertise `xhigh` in its CAPI capabilities, the CLI silently falls back to the model **default (`medium`)** instead of clamping to the model's highest supported level (`max`). The result is the opposite of intent: asking for *maximum* reasoning yields one of the *lower* levels — and it happens silently.

## Affected models

CAPI `/models` advertises a per-model `reasoning_effort` list. As observed on CLI 1.0.62 (enterprise endpoint):

| model | advertised efforts | supports `xhigh`? |
|---|---|---|
| `claude-opus-4.6` | low, medium, high, **max** | ❌ (ceiling = `max`) |
| `claude-sonnet-4.6` | low, medium, high, **max** | ❌ (ceiling = `max`) |
| `claude-opus-4.7` | low, medium, high, **xhigh**, max | ✅ |
| `claude-opus-4.8` | low, medium, high, **xhigh**, max | ✅ |

## Steps to reproduce

1. Configure reasoning effort `xhigh` (e.g. `effortLevel: "xhigh"` in `settings.json`, or via `/model`).
2. Run a request using `claude-opus-4.6` (or `claude-sonnet-4.6`).
3. Inspect the reasoning effort actually applied to the model call.

## Actual

The applied reasoning effort is `medium` (the model default). Verified across a large number of sessions: `xhigh`-configured requests on `claude-opus-4.6` are applied as `medium` — never `xhigh`, and never the model's real ceiling `max`. (The CLI's model-resolution still reflects the configured `xhigh`; only the effective model call is downgraded, which makes this hard to notice.)

## Expected

One of:
- **Clamp** an unsupported-but-higher request to the model's highest supported level (`xhigh` → `max`), or
- **Warn** that `xhigh` is unsupported for the selected model and report what was used instead.

Silently substituting the **default** (`medium`) for an explicit "maximum" request is surprising.

## Impact

- Users/automation setting `xhigh` for "maximum reasoning" actually get `medium` on these models.
- Fixed-`xhigh` model comparisons (e.g. evals) are invalid for models lacking `xhigh` — they run at a *lower* effort than peers, not an equal or higher one.
- The downgrade is silent, so it's easy to ship results under a false assumption.

## Open question

Is the fallback-to-default intentional, or should an unsupported level clamp to the nearest supported ceiling? Filing to start that discussion rather than assuming it's a bug.

## Environment

- Copilot CLI 1.0.62, Windows, enterprise plan.

Guia de contribuição

Abrir o guia de contribuição

Direção de pesquisa

Start at the `/model` entry point and the persisted `effortLevel` setting, then trace how the CAPI `/models` reasoning_effort list is used before the effective model call. Reproduce with a model that advertises `max` but not `xhigh`; done means the behavior follows an agreed clamp or warning policy and is covered by a regression test.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
shell
Domínio
cli
Tipo de issue
Bug
Dificuldade
3/5
Tempo estimado
1-2 dias
Status de atividade
Pouca atividade
Clareza
Razoavelmente clara
Facilidade para iniciantes
55/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.