anthropics / anthropics/claude-code

[BUG] Embedded CLAUDE.md intermittently reaches the API with one multibyte character replaced by three U+FFFD, changing the prompt prefix mid-session and defeating prompt caching (2.1.258, Windows)

オープン
#93,848 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
area:core bug has repro platform:windows
主要言語
Python
スター
145k
フォーク
23.1k
PR マージ指標
PR 指標を取得中

説明

## Summary

In headless sessions (`claude -p`) on Windows, the text of the user's global `CLAUDE.md` that Claude Code embeds in the first user message (`messages[0].content[0].text`, inside the `` "Codebase and user instructions are shown below") intermittently arrives at the API with **one multibyte character replaced by three U+FFFD replacement characters**. The file on disk is valid UTF-8 and never changes. In the affected requests the prompt prefix therefore differs from the other requests of the same session, so every such request is a full prompt-cache miss for the whole conversation.

## Environment

- Claude Code 2.1.258 (native Windows build), Windows 11 22H2 (10.0.22621), PowerShell 7 host
- Global instructions file: `%USERPROFILE%\.claude\CLAUDE.md`, 19,597 bytes, UTF-8 without BOM, Korean prose; `bytes.decode('utf-8')` in strict mode succeeds
- Session form: `claude -p` child sessions with `--allowedTools` (used as workers), API traffic observed through a local proxy set with `ANTHROPIC_BASE_URL` that records `system`, `messages[0]` and the tool names of every outgoing request

## What I observed

Over one 20-turn session, the outgoing `messages[0].content[0].text` came in two variants. They differ in exactly one place:

- clean variant (14 of 17 requests): `'바람이 불어 창문이 흔들리는 탓에 …'`
- corrupted variant (3 of 17 requests): `'바람이 ���어 창문이 흔들리는 탓에 …'`

`difflib` opcodes between the two texts: `[('replace', 3615, 3616, 3615, 3618)]`, i.e. the single character `불` (UTF-8 `EB B6 88`) became three replacement characters. Nothing else in the block, the `system` array, or the tool list differs.

Facts about the position:

- character index 3615 of the 19,136-character block; UTF-8 byte offset 7,982 inside the block; byte offset 7,665 inside the file
- neither offset is a 4 KiB / 8 KiB / 16 KiB boundary
- the position is identical in every corrupted request across four separate sessions (two probe runs plus two long worker runs); only *whether* a given request is corrupted varies
- the corrupted variant appeared both as the very first request of a session and in later requests (for example requests 2, 11 and 32 of one session; 15, 24 and 26 of another), with the clean variant in between, and the file was not modified during any session

Per-request view from the proxy log of one session (`sid` = first 8 hex chars of `sha256(system + messages[0])`, `in` = total input tokens, `cached` = cached input tokens reported by the API):

```
msgs sid in cached
5 600ce64a 58662 0 <- first big request
7 600ce64a 63738 58659
9 600ce64a 63807 38874
11 600ce64a 68896 63735
13 600ce64a 68964 68893
15 8fdae59a 74059 0 <- corrupted variant: different prefix, full miss
17 600ce64a 74127 63804
19 600ce64a 79170 68961
21 600ce64a 79239 74124
24 8fdae59a 84380 74056 <- corrupted variant again
26 8fdae59a 84448 0 <- full miss
28 600ce64a 89476 79167
```

## Impact

Each corrupted request changes the prefix of the whole conversation, so the request misses the prompt cache for the entire context (60-100k tokens here). With the corruption on about a fifth of the requests, the sessions read 75-76% of their input from cache; with the proxy substituting the clean text for the corrupted one (a workaround, see below), the same task reads 91.6% from cache and the uncached token volume drops from about 300k to 93k. Besides cost, a request that carries `�` in the instructions is also a small fidelity loss.

## Steps to reproduce

1. Put a UTF-8 `CLAUDE.md` with a few KB of Korean (or other multibyte) prose in `%USERPROFILE%\.claude\` on Windows.
2. Point `ANTHROPIC_BASE_URL` at a local proxy that dumps `messages[0]` of every request (any pass-through proxy works).
3. Run `claude -p "" --allowedTools Read,Grep,Bash`.
4. Diff `messages[0].content[0].text` across the requests: in my runs 3 of 17 requests carried the corrupted character at the same position; the rest were byte-identical to the file.

I could not find the pattern that decides which request is corrupted; the file, its mtime and the session were unchanged between a clean and a corrupted request seconds apart.

## Related

- #40396 (closed) reported U+FFFD for Korean characters in responses on macOS; this report is about the *request* side and the embedded instructions file, on Windows.
- #88836 reports U+FFFD in AskUserQuestion since 2.1.235; possibly the same decoding path.

## Workaround

A proxy that remembers each clean first message it has seen and, when a request differs from a remembered text only by runs of U+FFFD, sends the remembered text instead. That keeps the prefix stable but should not be needed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

Reproduce with a UTF-8 Korean %USERPROFILE%\.claude\CLAUDE.md, claude -p, --allowedTools, and a proxy recording messages[0].content[0].text. Trace how the file becomes the embedded system-reminder text and compare clean and corrupted requests, checking the related U+FFFD reports in #40396 and #88836. Done means repeated requests preserve the original character and prompt-cache prefix.

索引モデルが issue の本文から書いたものです。

評価

領域
backend, cli
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
活発
明瞭さ
おおむね明確
初心者へのやさしさ
45/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。