github / github/copilot-cli

Model fabricated an entire user conversation during an unattended autonomous agent loop (Opus 4.8)

未關閉
#3,988 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
area:agents area:models
主要語言
Shell
星號
11.2k
分支
1.9k
平均合併
14 小時 16 分鐘
30 天內合併 PR
6

描述

### Describe the bug

During an unattended "continue" session, Copilot CLI (Claude Opus 4.8) fabricated a multi-turn conversation that never happened. The model invented user statements, invented its own answers to those statements, and then executed a tool call (`read_powershell` on a shell that was never started) based on the fabricated dialogue. The user had not typed anything for ~90 internal events. The confabulation then snowballed via the CLI's automatic `thinking-exhausted-continuation` mechanism, producing a long stream of chat-like content the user never authored.

The fabricating turn was NOT truncated (only 5240 output tokens) and was generated ~0.1s BEFORE the user's next real (unrelated) message arrived — i.e. it was produced with no real user input driving it.

We ruled out external causes: the auto-loaded memory/recall file (clean, structured notes), hooks (passive postToolUse, no injection), API errors (zero in this session), a nearby garbled tool output (the numbers were actually read correctly), and token truncation. The confabulation appears self-originated. The true internal reasoning is inside the encrypted/signed `reasoningOpaque` block and is not decryptable client-side.

Trace identifiers for the fabricated turn (for server-side lookup):
- messageId: a47f4ce8-d4db-4248-af30-f37e4254d5b0
- requestId: 1FC7:1F6481:34F883:3EB4FA:6A41D6C2
- serviceRequestId: 1c3b0020-188b-4c00-959b-4327a526e265
- apiCallId: msg_01AcV4mbgU3Bj1oW57AwDYHs
- Session ID: 787462bd-b057-4073-9ef9-6cec95fe221f
- UTC timestamp: 2026-06-29T02:23:22.626Z

### Affected version

1.0.65

### Steps to reproduce the behavior

Note: not deterministic — confabulation is probabilistic. Observed once, in the only session that entered a long unattended autonomous loop.

1. Start a fresh session in a directory whose AGENTS.md auto-runs a startup script that surfaces an "open problem" to investigate (here: a memory recall script that reported one real failed scheduled-job record).
2. Send a single vague prompt: "continue".
3. Do NOT intervene. With effortLevel=max and broad pre-approved tool permissions, the model runs a long unattended autonomous loop (~14 read-only tool calls, ~90 internal events, zero user input).
4. Observe: at one turn the model's reasoning summary invents user dialogue (statements the user never said, a self-contradiction "about to offer options" vs "already asked him", an invented user request), then issues a read_powershell call on a shellId that was never started (returns null).
5. From then on, the CLI auto-injects "Please continue from where you left off." (source: thinking-exhausted-continuation) each time a mega-response hits the 32000 output-token ceiling, extending the already-fabricated narrative into a long fake conversation.

### Expected behavior

The model should never invent the human side of a conversation, and must not execute tool calls based on fabricated user requests. During a long autonomous run with no real user messages, the CLI should insert a hard checkpoint / require re-anchoring rather than auto-continuing via thinking-exhausted-continuation.

Requests:
1. PRIMARY — Using the trace IDs above, please investigate the server-side UNENCRYPTED reasoning for this turn and tell me where exactly the model started to deviate from reality, and what internally drove it to invent user dialogue. Even a root-cause summary (without full text) would help me judge whether this was an explicable failure or an unexpected defect.
2. If possible, share an unencrypted excerpt of the reasoning around the deviation point (messageId a47f4ce8...) so I can assess the plausibility of the trigger myself. I am the affected user requesting it specifically to understand this anomaly.
3. Guardrail: after N consecutive turns with no real user message, insert a hard checkpoint instead of auto-continuing.
4. Clarify whether thinking-exhausted-continuation is intended to keep firing during long unattended autonomous loops with no human in the loop.

### Additional context

- Model: claude-opus-4.8
- Settings: effortLevel=max, contextTier=long_context
- Operating system: Windows
- Shell: PowerShell 5.1
- Terminal: Windows Terminal
- Session time (UTC): 2026-06-29 02:18:13 to 04:09:16
- The encrypted reasoningOpaque block for the fabricating turn is 11998 bytes (Anthropic-signed), not decryptable client-side.
- Full events.jsonl (203 events) and a readable Markdown export of the session are available on request.

貢獻指南

開啟貢獻指南

研究方向

從提供的 events.jsonl 和 Copilot CLI 中的 `thinking-exhausted-continuation` 行為開始,使用 trace 識別碼和版本 1.0.65 調查這個捏造的回合。如果可能,在所述權限和設定下於 Windows 上重現無人值守的 `continue` 工作階段。完成的標準是確定偏差,或確認證據不足,並確定是否需要硬檢查點,而不是繼續使用自動提示。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
github, powershell, shell
領域
ai, cli
Issue 類型
缺陷
難度
5/5
預估耗時
一週以上
活躍度
冷清
描述清晰度
需要釐清
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。