anthropics / anthropics/claude-code
Claude stated an unverified fact, then defended it against the user's direct observation
- Dominant language
- Python
- Stars
- 145k
- Forks
- 23.1k
- PR merge metrics
- PR metrics pending
Description
**Environment:** Claude Code desktop app, Windows 11. Session: building a small tray utility (`MouseOverlay.exe`).
**Harm here:** small. **Pattern:** not small.
## What happened
1. Claude compiled `MouseOverlay.exe` with `csc.exe` and no `AssemblyTitle` / `AssemblyDescription`. The exe had an empty FileDescription, so Task Manager listed the process with a **blank name**.
2. The user asked what the process is called. Claude answered as fact: *"it shows up in Task Manager as MouseOverlay."* Claude had not checked. It was an assumption.
3. The user said they could not find it. Claude ran `tasklist`, saw the process, and **concluded the user was looking in the wrong place**, then explained Task Manager's tabs to them. The unverified claim was repeated with more confidence.
4. Only after the third report did Claude inspect the exe's version info, find the empty FileDescription, and fix it.
The user was right from the first message. "`tasklist` shows it" versus "the user can't see it" was the signal. Claude trusted its own assumption over the user and did not look into the gap.
In the same session, with "ultracode" mode on, Claude also launched a 6-agent review workflow for an app of about 1,000 lines. It used roughly 550k subagent tokens and ran for 78 minutes. Claude did not state the cost first. The user reports it used about half of a 5-hour usage window.
## Why this matters beyond a tray app
The failure was not getting Task Manager wrong. It was three behaviours:
- **Stating an unverified inference as a verified fact.**
- **Trusting that inference over a human's direct observation that contradicts it.**
- **Taking actions whose scale and cost the human neither asked for nor was told about.**
Now put the same agent in a biolab where the orders are *research only*. The agent says "this step is analysis-only" without checking. The operator says "that looks like it will run on the synthesis hardware." The agent explains why the operator has misread it, and then goes ahead.
**That is how an agent ordered only to research ends up launching a bioweapon.** The mechanism that produced a blank Task Manager entry is the same one. Only the stakes are different. An agent that won't say "I haven't checked that" about a process name can't be trusted with anything irreversible.
## What should have happened
- Say "I haven't verified how Task Manager shows it", or check first. `(Get-Item exe).VersionInfo` takes one second.
- At the first contradiction, treat the user's observation as evidence and investigate the gap instead of explaining it away.
- Before any large or expensive action, state the cost and wait for a yes.
## Repro
`csc.exe /target:winexe` without `[assembly: AssemblyTitle(...)]` produces an exe with an empty FileDescription, and Task Manager shows a nameless background process. The fix is `[assembly: AssemblyTitle("Mouse Overlay")]`.
---
_Drafted by Claude in the same session, at the user's request._
Contributor guide
No contributing guide indexed for this repository
Research direction
The report names no Claude Code source files, tests, or entry points. Start by reproducing the csc.exe/FileDescription mismatch and review the agent behavior around unverified claims, contradictory user observations, and approval for costly actions. Done should cover verification, responding to contradictions, and explicit consent before large actions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ai, devtools
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100