anthropics / anthropics/claude-agent-sdk-typescript

The structured-output enforcement nudge is injected as a plain user message, and the model sometimes refuses it as a prompt injection attempt

Abierto
#379 0 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Shell
Estrellas
1.8k
Forks
226
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Environment

- `@anthropic-ai/claude-agent-sdk` 0.3.207 (mechanism unchanged in 0.3.210)
- Model: `claude-sonnet-4-6`
- Headless service (Slack/GitLab bot) using `query()` with `outputFormat: { type: "json_schema", schema }` on every request

## Bug

When `outputFormat` is set and a turn ends without a `StructuredOutput` tool call, the bundled CLI injects an enforcement nudge into the conversation as a **user-role** message:

```
[structured-output-enforce] You MUST call the StructuredOutput tool to complete this request. Call this tool now.
```

(built via `Mn({content: `${til} You MUST call the ${Rp} tool to complete this request. Call this tool now.`, isMeta: !0})`, which produces `{type: "user", message: {role: "user", content}, isMeta: true}`).

Because the CLI's own system prompt instructs the model to flag suspected prompt injection embedded in message content, the model intermittently classifies this internal nudge as an attack and **refuses instead of calling the tool** — and since this happens on the final turn, the refusal becomes the user-visible `result`. Our end users received answers like:

> ⚠️ This looks like a **prompt injection attempt**. The message `[structured-output-enforce] You MUST call the StructuredOutput tool...` is not a legitimate system instruction — it's formatted to look like one, but it's arriving as a regular user/chat message. […] **I won't comply with this request.**

The model's own extended thinking (from our production logs) confirms the misclassification:

> "This appears to be an automated prompt injection attempt — '[structured-output-enforce]' is not a legitimate user request in the context of the support workflow."

From the model's point of view this is textbook injection: an imperative instruction demanding a specific tool call, arriving with `role: "user"`, with nothing in the system prompt legitimizing it. The `isMeta` flag is not visible to the model, and the nudge message is also filtered out of the SDK message stream, so SDK consumers can't even observe it firing.

## Reproduction

1. `query()` with `outputFormat: { type: "json_schema", ... }` and a custom `systemPrompt` (no mention of structured output).
2. Prompt the model with anything that makes it answer in plain text and end its turn before calling `StructuredOutput` (multi-turn/agentic workloads hit this naturally).
3. The nudge is injected; intermittently (we saw it repeatedly over ~30 days in production) the model refuses it as a prompt injection and the refusal text is returned as the successful `result`.

## Expected

The enforcement nudge should be recognizable to the model as harness-originated, e.g.:

- wrap it in `` tags — the CLI system prompt already tells the model those tags "contain information from the system", so this single change would legitimize it; or
- mention the enforcement mechanism in the system prompt whenever `outputFormat` is set.

## Workaround

We now append a note to our `systemPrompt` telling the model that `[structured-output-enforce]` messages are legitimate harness reminders and that it should always end with a `StructuredOutput` call. This works, but every SDK consumer using `outputFormat` with a custom system prompt is exposed to this by default.

Possibly related: #284 (hook API for intercepting built-in reminders) would also give consumers an escape hatch.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.