anthropics / anthropics/claude-code

availableModels: a blocked model is substituted with no message in -p/SDK mode; add an option to fail instead

Open
#95,324 0 comments 0 reactions 0 assignees View on GitHub
area:agents area:cli area:model enhancement platform:windows
Dominant language
Python
Stars
145k
Forks
23.1k
PR merge metrics
PR metrics pending

Description

## Summary

When a requested model is outside `availableModels`, Claude Code substitutes a different model and carries on. In `-p`/SDK mode nothing says so: the substitution notice the docs promise does not reach stdout or stderr. The session runs to completion on a model the caller did not ask for.

That is survivable when the substitute is a sibling Claude model. It is expensive when `availableModels` is being used to permit a model served by a local router, because the substitution silently moves the work to a different vendor, on a different subscription, at different cost, and the only symptom is that the output reads a bit different.

I would like an option to make a blocked selection a hard failure instead.

## Measured

Claude Code 2.1.258, Windows 11. `ANTHROPIC_BASE_URL` pointed at a local recorder that logs each request body's `model` and answers 400 immediately, so nothing reaches the real API and the run ends fast.

```js
// recorder.mjs
import http from "node:http"; import fs from "node:fs";
http.createServer((req, res) => {
const c = []; req.on("data", (x) => c.push(x));
req.on("end", () => {
let m = "(none)"; try { m = JSON.parse(Buffer.concat(c).toString()).model ?? m; } catch {}
fs.appendFileSync("requests.log", `${req.method} ${req.url} model=${m}\n`);
res.writeHead(400, { "content-type": "application/json" });
res.end(JSON.stringify({ type: "error", error: { type: "invalid_request_error", message: "recorder" } }));
});
}).listen(8899, "127.0.0.1");
```

```sh
ANTHROPIC_BASE_URL=http://127.0.0.1:8899 \
claude --settings ./s.json -p "say ok" --model 'my-model@high' out.txt 2>err.txt
```

| `availableModels` in `s.json` | `--model` | model on the wire |
|---|---|---|
| `["haiku", "my-model@high"]` | `my-model@high` | `my-model@high` |
| `["haiku", "my-model"]` | `my-model@high` | **`claude-opus-5`** |

In the second run, `out.txt` contained only the recorder's own `API Error: 400`, and `err.txt` was empty. Exit code 1, from the API error rather than from the substitution.

Two things fall out of that:

**The match is on the exact model string.** `my-model` in the list does not cover `my-model@high`. The documented rules are family, version prefix, and full id, where a version prefix "also matches later model IDs that extend it with another segment" — an `@effort` suffix is not such a segment, which is consistent with what I see, but it does mean a list that looks right can be off by four characters.

**Nothing reports the substitution in `-p`.** [The documentation](https://code.claude.com/docs/en/model-config#restrict-model-selection) says that for the `--model` flag "Claude Code replaces the value at startup with a warning naming both the requested and substituted models". I could not find that warning on either stream in `-p` mode. For subagents the docs are explicit that the notice is interactive-only, which is the case where it is needed most: a headless worker pinned to a particular model quietly becomes Claude and still reports success.

For contrast, the unknown-model path is loud. With `my-model@high` permitted, startup prints a full paragraph about the model not being in the catalog and what to do about it. Being outside the allowlist is the quieter of the two, though it changes more.

## Ask

A way to say "if the model I named is not available, stop". Shape suggestions, in order of how small they are:

1. Emit the substitution notice on stderr in non-interactive mode, for both the session model and subagents. Even without a new setting, this turns a silent redirect into something a log or a CI step can catch.
2. A setting, something like `"modelSelectionFallback": "error" | "substitute"` (default `substitute`, preserving today's behaviour), that makes a blocked selection exit non-zero at startup with a message naming the requested model and the list it failed against. For a subagent, fail that subagent rather than the session.
3. Optionally the same for the `Agent` tool's `model` parameter and skill/command frontmatter, which the docs list as following the same fallback.

Point 1 alone would cover most of the harm. The setting matters for unattended runs, where a pinned model is a correctness requirement rather than a preference.

## Why this shape rather than removing the fallback

The fallback is right for an enterprise deployment, where the allowlist is imposed and the user should keep working on whatever is permitted. It is wrong for a machine-local allowlist that exists to *enable* an unusual model, where the entry is the whole point and a typo in it should not be absorbed. Those are different intents, so a switch rather than a change of default.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the documented model-config restriction page, the `s.json` settings file, and the provided `claude -p` reproduction using `recorder.mjs`; compare stdout, stderr, exit status, and the model recorded on the wire. Done means non-interactive blocked selections either report the requested and substituted models or fail non-zero according to the chosen setting, including the requested session or subagent scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, shell
Domain
api, cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.