anthropics / anthropics/claude-code
[BUG] Claude Code CLI: `--json-schema` calls emit literal `$PARAMETER_NAME` placeholder keys on toolless calls
- Lenguaje dominante
- Python
- Estrellas
- 145k
- Forks
- 23.1k
- Métricas de merge de PR
- Métricas de PR pendientes
Descripción
### Preflight Checklist
- [x] I have searched [existing issues](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20label%3Abug) and this hasn't been reported yet
- [x] This is a single bug report (please file separate reports for different bugs)
- [x] I am using the latest version of Claude Code
### What's Wrong?
**Version:** 2.1.233 (Claude Code), Linux x86-64
**Impact:** wasted round trips (~27% of affected call class), self-healing, no data loss
## Summary
On a headless `--json-schema` (structured output) call where **no other tools are enabled**, the
model's first `StructuredOutput` invocation frequently emits keys named with the literal
tool-call template placeholders `$PARAMETER_NAME` / `$PARAMETER_NAME2` instead of the schema's
real field names. It fails validation, the CLI retries internally, and the model self-corrects
on the next attempt.
The values are correct — only the keys are wrong. The model clearly knows what it wants to say
and mis-names the fields it says it in:
```json
{"$PARAMETER_NAME":"statement"}
{"$PARAMETER_NAME":"statement","$PARAMETER_NAME2":"placeholder"}
{"$PARAMETER_NAME":"statement","$PARAMETER_NAME2":"Exploiting this grants no new capability only if the party supplying the ciphertext already had the ability to crash the decrypting process; in the documented ABE deployment ... an external, unprivileged attacker gains a previously unavailable capability: reliably crashing the decrypting process via a malformed but unauthenticated ciphertext."}
```
That third payload is the tell: a fully-formed, on-task answer for the schema's second field,
emitted under a placeholder key.
## The discriminator is tool availability, not schema shape
Measured over 131 CLI session transcripts from a single automated run, classified by whether the
session had filesystem tools available:
| call class | sessions | with placeholder keys | rate |
|---|---|---|---|
| toolless (`--json-schema` only) | 71 | 19 | **26.8%** |
| read-capable (fs tools + `--json-schema`) | 60 | 0 | **0.0%** |
Fisher exact, two-sided: **p = 4.4e-06**.
Schema size and complexity are ruled out as the cause. Within the affected class, an 8-field
/ 3363-byte schema and a 4-field / 2964-byte schema show comparable rates, while the *largest*
schema observed (3891 bytes) never reproduces it — it is used only on read-capable calls.
Method note: sessions were classified by whether the transcript shows fs-tool usage
(`mcp__*-fs-tools`, `Read`, `Grep`). A per-call classification against the actual
`allow_read` flag may shift the exact rates; the categorical split (nonzero vs zero) is stable.
## Relationship to existing reports
This looks like the same underlying failure as two open issues — the model falling back to the
generic tool-call scaffolding instead of the concrete schema — but with a different manifestation
and, importantly, a **different trigger correlate**:
| | #49747 | #76901 | this report |
|---|---|---|---|
| what leaks | legacy **XML structure** (``) into JSON args | same XML-in-string leak, during StructuredOutput retries | the template's **placeholder identifiers** as JSON keys |
| surface | custom MCP tools, 5–7 required params | Workflow `agent()` `schema:` | CLI `--json-schema` / `StructuredOutput` |
| trigger correlate | **longer** argument payloads | retry exhaustion → degenerate stub accepted | **tool availability** (see table above) |
| version | 2.1.112 | 2.1.207 | 2.1.233 |
`$PARAMETER_NAME` is the placeholder *inside* `` in that same
template — so #49747 leaks the template's structure and this leaks its identifiers.
**The length correlation does not reproduce here; it inverts.** Among toolless sessions, the
placeholder-affected ones produced *shorter* payloads than the clean ones (median 2817 vs 8482
chars, n=19 vs 52). Caveat: schema and activity are a confound — the affected calls use smaller
schemas — so this is offered as "#49747's length trigger is not what is happening here", not as a
claim that brevity causes it.
The tool-availability discriminator appears to be new information not present in either issue.
## Suspected mechanism
`$PARAMETER_NAME` / `$FUNCTION_NAME` are the placeholder tokens of the generic tool-call
template. The string does **not** appear in the calling application's source, and `strings` over
the 2.1.233 binary finds **0** occurrences of `PARAMETER_NAME` (against 31 for
`StructuredOutput`) — so it is not being injected by the harness or by the CLI's own prompt
text. The likely reading is that with no concrete tool definitions in context to anchor on, the
model falls back to emitting the template literally.
Two CLI-side details make this reachable rather than prevented:
- `StructuredOutput`'s base `inputSchema` is an empty passthrough; `--json-schema` is enforced
**post hoc** (ajv) rather than as a decoding constraint, so a structurally-wrong-but-valid-JSON
payload is generated first and rejected after.
- Constrained decoding appears to sit behind the remote gate `tengu_structured_output_strict`
(4 occurrences in the 2.1.233 binary), which is off by default with no flag or environment
override exposed.
## Impact
Low but nonzero, and invisible to the caller. In the measured run: 19 extra round trips, roughly
$0.69–1.00 of $28.72 total spend (~2.4–3.5%). It is self-healing and cost us no results.
Notably the retry happens *inside* the CLI, so an application's own
"retry on invalid structured output" counter never fires and never sees it — the only evidence is
in the session transcript. Anyone measuring structured-output reliability from the outside will
under-count this to zero.
## Suggested fixes
1. Enable constrained decoding for `--json-schema` by default, or expose a flag/env override for
`tengu_structured_output_strict` so callers who need determinism can opt in.
2. Failing that, include the concrete schema field names in the `StructuredOutput` tool's
`inputSchema` rather than an empty passthrough, so there is something to anchor on when no
other tools are present.
3. Surface intra-CLI structured-output retries in the session output (a counter or a stderr note),
so callers can measure this class of failure at all.
## Filing note
Related to (but distinct from) **anthropics/claude-code#49747** and **#76901** — same family,
different manifestation and a different trigger correlate. Worth cross-referencing both.
## Adjacent, larger signal
In the same dataset only **30 of 131** structured calls validated on the first attempt, costing an
estimated **$7.38 (25.7%)** in retries overall. The bulk of that is *not* the placeholder bug —
those call classes never emit placeholders — so first-attempt schema conformance on
`--json-schema` looks worth investigating independently of this report.
### Is this a regression?
I don't know
### Last Working Version
_No response_
### Claude Code Version
2.1.233 (Claude Code)
### Platform
Anthropic API
### Operating System
Linux
### Terminal/Shell
N/a
### Additional Information
_No response_
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
The payload names no repository files or tests. Start at the headless `--json-schema` and `StructuredOutput` entry points, reproducing toolless and read-capable calls to compare placeholder-key behavior. Done means the reported toolless calls no longer emit literal placeholder keys or require the hidden retry described here.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- cli, python
- Área
- api, cli
- Tipo de issue
- Error
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Estado de actividad
- Activo
- Claridad
- Bastante claro
- Aptitud para principiantes
- 35/100