Model fabricated an entire user conversation during an unattended autonomous agent loop (Opus 4.8)
- 主要语言
- Shell
- 星标
- 11.2k
- 派生
- 1.9k
- 平均合并
- 14 小时 16 分钟
- 30 天内合并 PR
- 6
描述
### Describe the bug
During an unattended "continue" session, Copilot CLI (Claude Opus 4.8) fabricated a multi-turn conversation that never happened. The model invented user statements, invented its own answers to those statements, and then executed a tool call (`read_powershell` on a shell that was never started) based on the fabricated dialogue. The user had not typed anything for ~90 internal events. The confabulation then snowballed via the CLI's automatic `thinking-exhausted-continuation` mechanism, producing a long stream of chat-like content the user never authored.
The fabricating turn was NOT truncated (only 5240 output tokens) and was generated ~0.1s BEFORE the user's next real (unrelated) message arrived — i.e. it was produced with no real user input driving it.
We ruled out external causes: the auto-loaded memory/recall file (clean, structured notes), hooks (passive postToolUse, no injection), API errors (zero in this session), a nearby garbled tool output (the numbers were actually read correctly), and token truncation. The confabulation appears self-originated. The true internal reasoning is inside the encrypted/signed `reasoningOpaque` block and is not decryptable client-side.
Trace identifiers for the fabricated turn (for server-side lookup):
- messageId: a47f4ce8-d4db-4248-af30-f37e4254d5b0
- requestId: 1FC7:1F6481:34F883:3EB4FA:6A41D6C2
- serviceRequestId: 1c3b0020-188b-4c00-959b-4327a526e265
- apiCallId: msg_01AcV4mbgU3Bj1oW57AwDYHs
- Session ID: 787462bd-b057-4073-9ef9-6cec95fe221f
- UTC timestamp: 2026-06-29T02:23:22.626Z
### Affected version
1.0.65
### Steps to reproduce the behavior
Note: not deterministic — confabulation is probabilistic. Observed once, in the only session that entered a long unattended autonomous loop.
1. Start a fresh session in a directory whose AGENTS.md auto-runs a startup script that surfaces an "open problem" to investigate (here: a memory recall script that reported one real failed scheduled-job record).
2. Send a single vague prompt: "continue".
3. Do NOT intervene. With effortLevel=max and broad pre-approved tool permissions, the model runs a long unattended autonomous loop (~14 read-only tool calls, ~90 internal events, zero user input).
4. Observe: at one turn the model's reasoning summary invents user dialogue (statements the user never said, a self-contradiction "about to offer options" vs "already asked him", an invented user request), then issues a read_powershell call on a shellId that was never started (returns null).
5. From then on, the CLI auto-injects "Please continue from where you left off." (source: thinking-exhausted-continuation) each time a mega-response hits the 32000 output-token ceiling, extending the already-fabricated narrative into a long fake conversation.
### Expected behavior
The model should never invent the human side of a conversation, and must not execute tool calls based on fabricated user requests. During a long autonomous run with no real user messages, the CLI should insert a hard checkpoint / require re-anchoring rather than auto-continuing via thinking-exhausted-continuation.
Requests:
1. PRIMARY — Using the trace IDs above, please investigate the server-side UNENCRYPTED reasoning for this turn and tell me where exactly the model started to deviate from reality, and what internally drove it to invent user dialogue. Even a root-cause summary (without full text) would help me judge whether this was an explicable failure or an unexpected defect.
2. If possible, share an unencrypted excerpt of the reasoning around the deviation point (messageId a47f4ce8...) so I can assess the plausibility of the trigger myself. I am the affected user requesting it specifically to understand this anomaly.
3. Guardrail: after N consecutive turns with no real user message, insert a hard checkpoint instead of auto-continuing.
4. Clarify whether thinking-exhausted-continuation is intended to keep firing during long unattended autonomous loops with no human in the loop.
### Additional context
- Model: claude-opus-4.8
- Settings: effortLevel=max, contextTier=long_context
- Operating system: Windows
- Shell: PowerShell 5.1
- Terminal: Windows Terminal
- Session time (UTC): 2026-06-29 02:18:13 to 04:09:16
- The encrypted reasoningOpaque block for the fabricating turn is 11998 bytes (Anthropic-signed), not decryptable client-side.
- Full events.jsonl (203 events) and a readable Markdown export of the session are available on request.
贡献指南
调研方向
从提供的 events.jsonl 和 Copilot CLI 中的 `thinking-exhausted-continuation` 行为开始,使用 trace 标识符和版本 1.0.65 调查这个捏造的回合。如果可能,在所述权限和设置下于 Windows 上复现无人值守的 `continue` 会话。完成的标准是确定偏差,或确认证据不足,并确定是否需要硬检查点,而不是继续使用自动提示。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- github, powershell, shell
- 领域
- ai, cli
- Issue 类型
- 缺陷
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 冷清
- 描述清晰度
- 需要澄清
- 新手友好度
- 25/100