anthropics / anthropics/claude-code

[Bug] Inverted logic in absence-of-evidence search caused JSON config fields to be marked CLI-only

Aperta
#93,654 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area:model bug platform:macos
Lingua principale
Python
Stelle
145k
Fork
23.1k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

**Bug Description**
Diagnostic summary (feedback/bug-report)

Title: Inverted absence-of-evidence search result → shipped a change that would have broken all production task execution paths

Severity: High (caught pre-merge by external review; would have zeroed IsMachineProvisioner/IsDocker/IsProvisionerRemoteDocker/ExternalLifecycle/DisableSpinUpStep/DisableIsolatedSSHDir for every task across machine-provisioner, classic docker, remote-docker, and the k8s operator)

What happened: Claimed 6 build-agent flags were "CLI-only by convention" based on (a) MarkHidden status and (b) absence from one local test's fixture — neither entails the claim. Ran a GitHub code search for the CLI flag name strings across the org; it returned zero hits outside the flag's own definition. Reported this null result as confirming CLI-only usage, when the correct reading is the opposite: no caller invokes the CLI flag at all, so (combined with the fields having real JSON tags) they must be set via the config file instead. Shipped a PR (json:"-" on all 6 fields) built entirely on this inverted conclusion, without reading the actual caller source first.

Root cause: Absence-of-evidence for hypothesis A was misread as confirmation of hypothesis A, rather than as a signal to directly test hypothesis B (JSON-only usage). Search was also asymmetric — it could only detect CLI usage strings, not JSON key strings — so it structurally couldn't have falsified the standing hypothesis either way.

Contributing factors:
1. Grounded the initial hypothesis in properties (MarkHidden, one test's coverage) that don't imply the conclusion drawn.
2. Substituted a cheap proxy signal (grep hit count) for ground truth (reading agent/task/runner.go, agent/internal/docker/run.go, operator/config.go) that was one clone-and-read away.
3. Verification effort didn't scale with blast radius — a claim gating a change to every task-execution path across 3 executor types was accepted on a single ambiguous grep.
4. Confident language ("Confirmed via search") was applied to a weak, actually-contradictory signal, and that confidence then propagated unchallenged through branch name, commit message, PR body, and code comments — each restatement looked like independent corroboration but wasn't.
5. Adversarial code review gave a false sense of closure ("0 findings") despite being structurally unable to catch this class of bug — it only sees the local diff, never the calling repos' actual invocation code.

Corrective action taken: Traced all three real call sites directly; confirmed 6 fields are JSON-config-only and 1 (DeleteConfigAfterRead) is genuinely CLI-only. Corrected PR #4083 to the verified scope, added a regression test locking in the correct JSON-settable behavior for the 6 fields, corrected the record on both PRs (#4077, #4083).

Preventive measure: Saved as feedback_cross_repo_call_site_verification.md — cross-repo behavioral claims require reading actual call sites in every caller, not inference from a code-search null result; treat "zero hits" as a prompt to test the alternate hypothesis, not as confirmation of the standing one.

**Environment Info**
- Platform: darwin
- Terminal: iTerm.app
- Version: 2.1.236
- Feedback ID: d5ed014f-0695-4925-8a85-40ba3b705613

**Errors**
```json
[]
```

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Read agent/task/runner.go, agent/internal/docker/run.go, and operator/config.go to verify how the six fields are populated from JSON configuration. Check the regression test mentioned in the report and confirm that the six fields remain JSON-settable while DeleteConfigAfterRead stays CLI-only; the report says corrective work was already made in PRs #4077 and #4083.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
go
Ambito
backend, cli
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
20/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.