anthropics / anthropics/claude-code
[Bug] Inverted logic in absence-of-evidence search caused JSON config fields to be marked CLI-only
- Lingua principale
- Python
- Stelle
- 145k
- Fork
- 23.1k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
**Bug Description**
Diagnostic summary (feedback/bug-report)
Title: Inverted absence-of-evidence search result → shipped a change that would have broken all production task execution paths
Severity: High (caught pre-merge by external review; would have zeroed IsMachineProvisioner/IsDocker/IsProvisionerRemoteDocker/ExternalLifecycle/DisableSpinUpStep/DisableIsolatedSSHDir for every task across machine-provisioner, classic docker, remote-docker, and the k8s operator)
What happened: Claimed 6 build-agent flags were "CLI-only by convention" based on (a) MarkHidden status and (b) absence from one local test's fixture — neither entails the claim. Ran a GitHub code search for the CLI flag name strings across the org; it returned zero hits outside the flag's own definition. Reported this null result as confirming CLI-only usage, when the correct reading is the opposite: no caller invokes the CLI flag at all, so (combined with the fields having real JSON tags) they must be set via the config file instead. Shipped a PR (json:"-" on all 6 fields) built entirely on this inverted conclusion, without reading the actual caller source first.
Root cause: Absence-of-evidence for hypothesis A was misread as confirmation of hypothesis A, rather than as a signal to directly test hypothesis B (JSON-only usage). Search was also asymmetric — it could only detect CLI usage strings, not JSON key strings — so it structurally couldn't have falsified the standing hypothesis either way.
Contributing factors:
1. Grounded the initial hypothesis in properties (MarkHidden, one test's coverage) that don't imply the conclusion drawn.
2. Substituted a cheap proxy signal (grep hit count) for ground truth (reading agent/task/runner.go, agent/internal/docker/run.go, operator/config.go) that was one clone-and-read away.
3. Verification effort didn't scale with blast radius — a claim gating a change to every task-execution path across 3 executor types was accepted on a single ambiguous grep.
4. Confident language ("Confirmed via search") was applied to a weak, actually-contradictory signal, and that confidence then propagated unchallenged through branch name, commit message, PR body, and code comments — each restatement looked like independent corroboration but wasn't.
5. Adversarial code review gave a false sense of closure ("0 findings") despite being structurally unable to catch this class of bug — it only sees the local diff, never the calling repos' actual invocation code.
Corrective action taken: Traced all three real call sites directly; confirmed 6 fields are JSON-config-only and 1 (DeleteConfigAfterRead) is genuinely CLI-only. Corrected PR #4083 to the verified scope, added a regression test locking in the correct JSON-settable behavior for the 6 fields, corrected the record on both PRs (#4077, #4083).
Preventive measure: Saved as feedback_cross_repo_call_site_verification.md — cross-repo behavioral claims require reading actual call sites in every caller, not inference from a code-search null result; treat "zero hits" as a prompt to test the alternate hypothesis, not as confirmation of the standing one.
**Environment Info**
- Platform: darwin
- Terminal: iTerm.app
- Version: 2.1.236
- Feedback ID: d5ed014f-0695-4925-8a85-40ba3b705613
**Errors**
```json
[]
```
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Direzione di ricerca
Read agent/task/runner.go, agent/internal/docker/run.go, and operator/config.go to verify how the six fields are populated from JSON configuration. Check the regression test mentioned in the report and confirm that the six fields remain JSON-settable while DeleteConfigAfterRead stays CLI-only; the report says corrective work was already made in PRs #4077 and #4083.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- go
- Ambito
- backend, cli
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 20/100