Unknown-model fallback prompt contradicts exec_command schema (command array vs cmd string)

Summary

The unknown-model fallback in Codex CLI 0.151.0 pairs a legacy shell-style apply_patch invocation example ({"command":["apply_patch", "..."]}) with the current exec_command tool, which requires a cmd string.

The fallback also sets apply_patch_tool_type: None, so the example cannot be interpreted as an invocation of an available native apply_patch tool in that configuration. Codex does provide a shell-accessible apply_patch executable; the problem is that the fallback prompt does not explain that route with arguments matching the tool it actually exposes.

This report is about prompt/tool-schema drift, not a claim that every unknown model needs a native freeform tool or that shell-based editing is unsupported.

Environment and verification scope

• Affected CLI version investigated: 0.151.0.
• Source reference: rust-v0.151.0, commit 78c290807ce710180111df227df3b7a4fe845452.
• Reported setup: Qwen3.5 served through vLLM, using the Responses API and automatic tool calling, with fallback model metadata.
• The source-level mismatch has been confirmed by inspection. A controlled before/after model experiment has not been performed, and this report does not claim a deterministic failure rate for a particular model size, quantization, or vLLM version.

Confirmed source-level mismatch

1. Unknown models select this prompt and UnifiedExec

In codex-rs/models-manager/src/model_info.rs, model_info_from_slug() constructs fallback metadata with:

shell_type: ConfigShellToolType::UnifiedExec,
model_messages: Some(local_model_messages_for_slug(slug)),
apply_patch_tool_type: None,
use_responses_lite: false,

For ordinary unknown slugs, local_model_messages_for_slug() uses BASE_INSTRUCTIONS, which includes ../prompt.md. This applies when no matching model metadata or explicit instruction override supplies a different configuration.

2. The fallback prompt contains an obsolete argument shape

codex-rs/models-manager/prompt.md contains this line:

- Use the `apply_patch` tool to edit files (NEVER try `applypatch` or `apply-patch`, only `apply_patch`): {"command":["apply_patch","*** Begin Patch\\n*** Update File: path/to/file.py\\n@@ def example():\\n- pass\\n+ return 123\\n*** End Patch"]}

The relevant mismatch is the command key plus an argv array.

3. The exposed shell tool requires a different key and type

codex-rs/core/src/tools/handlers/shell_spec.rs defines exec_command with:

• cmd: string, required;
• additionalProperties: false;
• strict: false.

The receiver in codex-rs/core/src/tools/handlers/unified_exec.rs likewise requires:

#[derive(Debug, Deserialize)]
pub(crate) struct ExecCommandArgs {
    pub(crate) cmd: String,
    // Other fields omitted.
}

There is no command alias for cmd. Simply renaming command to cmd in the existing prompt example would therefore still be insufficient: an argv array is not a shell-command string.

4. The shell fallback exists; native-tool absence is not itself the bug

In codex-rs/core/src/tools/spec_plan.rs, native ApplyPatchHandler registration requires an environment and model_info.apply_patch_tool_type.is_some().

Separately, codex-rs/arg0/src/lib.rs provides the apply_patch executable alias and dispatches it to the patch implementation. The shell-accessible route is also explicitly described in #2646.

Thus the fallback can edit through:

model -> exec_command({"cmd": "apply_patch ..."}) -> shell-accessible apply_patch -> patch engine

The prompt should describe that route rather than leave a legacy shell argument example and ambiguously refer to an unavailable native tool.

Observed symptom and limits of the causal claim

The reported failed exec_command arguments had this form (command body abbreviated):

{"command": "cd /testbed && ..."}

This is inconsistent with the required cmd field and is compatible with the hypothesis that the legacy example is influencing argument-name generation.

However, the output alone does not establish that the model copied the key from the prompt, nor that it copied the string type from the schema. With XML-style model output, a tool-call parser can also affect the resulting JSON representation. Raw model output and a controlled prompt-only A/B comparison are needed to establish causality.

Also, not finding cmd in session_meta.base_instructions does not mean the model was never supplied the schema: the ordinary Responses path sends instructions and tool definitions separately in client.rs. This issue is not alleging missing schema serialization.

Reproduction / inspection steps

Deterministic source inspection (no model inference required)

1. Check out rust-v0.151.0.
2. Trace model_info_from_slug() and the ordinary unknown-slug branch of local_model_messages_for_slug().
3. Compare the apply_patch example in models-manager/prompt.md with create_exec_command_tool_with_environment_id() and ExecCommandArgs.
4. Check the native patch registration guard in spec_plan.rs.

These steps expose the incompatible example regardless of whether a particular model manages to ignore it.

Behavioral verification proposed, not yet completed

Run the same bounded read/edit task in fresh sessions with an unknown model slug, a functioning Responses-compatible provider, and unchanged model/server/sampling settings. Compare:

• the original fallback prompt;
• the same prompt with only the obsolete example removed;
• the same prompt with schema-consistent shell-fallback guidance.

Capture the outgoing tool definitions, rendered model prompt where available, raw model output, parsed arguments, and actual edit result. Repeated runs are needed; a single successful run would not prove the issue is resolved.

Expected behavior and suggested fix

The fallback instructions should agree with the actual tool registry:

• When native apply_patch is exposed, instruct the model to use its declared input format.
• When native apply_patch is absent but shell execution and the helper are available, explain exec_command -> apply_patch using the cmd string.
• Do not describe either route as available when the session does not expose it.

A minimal fix is to remove the obsolete JSON example and explicitly distinguish the native tool from the shell executable. A more robust fix would derive this guidance from the resolved tool configuration.

For illustration, valid arguments to exec_command for a POSIX-shell fallback are:

{
  "cmd": "apply_patch <<'PATCH'\n*** Begin Patch\n*** Add File: probe.txt\n+hello\n*** End Patch\nPATCH"
}

This is an illustrative POSIX example, not a universal PowerShell instruction. The fix should not force unsupported custom/freeform tools onto every third-party provider or bypass existing permission checks.

Suggested regression coverage: verify that any fallback invocation example matches the registered tool schema, exercise an edit through the shell-only path, and preserve the native-tool path for models that support it.

Related reports

• #30648: prompt mentions native apply_patch while the reported tool list lacks it.
• #14046: fallback model metadata / patch-instruction regression for Azure-hosted GPT-4.1.
• #33405: broader third-party-provider native editing compatibility.

This report focuses on the narrower, source-confirmed command array versus cmd string mismatch in the unknown-model fallback prompt; it does not assume all of those reports have the same cause.

Start in codex-rs/models-manager/model_info.rs and prompt.md, then compare the fallback guidance with the exec_command schema in core/src/tools/handlers/shell_spec.rs and unified_exec.rs. Inspect native patch registration in core/src/tools/spec_plan.rs and the shell alias in arg0/src/lib.rs. Done means fallback examples match the registered schema, shell-only editing is covered by regression tests, and the native-tool path remains intact.