anthropics / anthropics/claude-code-action
Workflow-validation skip reports conclusion: success — indistinguishable from a real pass (follow-up to #1343)
- Vorherrschende Sprache
- TypeScript
- Sterne
- 8.9k
- Forks
- 2.1k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
Follow-up to #1343, which unified the two validation-skip code paths on `success`. That fixed the inconsistency, but the chosen value has a security-relevant side effect: **a skipped review is now indistinguishable from a passing review** — including to branch protection.
#1343 itself proposed the safer alternative in its final paragraph ("both paths could surface as `skipped` (or `neutral`)"). This issue argues that alternative should be preferred.
## Observed
Confirmed on `claude-code-action@v1`, private repo, August 2026 — both the first-add and modification paths now return `success`:
| Run | PR changed | Duration | Conclusion | Log contains |
|---|---|---|---|---|
| `31280931811` | adds a workflow | 14s | `success` | `Exiting due to workflow validation skip` |
| `31448510042` | **modifies** `claude-review.yml` | 16s | `success` | `Exiting due to workflow validation skip` |
For contrast, a genuine review run on the same repo takes ~2m39s–5m39s and reports real `num_turns` / `total_cost_usd`.
## Why `success` is the wrong value here
The anti-tamper check itself is sound — a PR should not be able to rewrite the reviewer that gates it. The problem is only the conclusion it reports.
**Any PR that edits the invoking workflow gets a green `claude-review` check with nothing reviewed — including a PR that weakens or disables the reviewer.** The check is structurally incapable of failing on exactly the PRs where review matters most.
Because the conclusion is `success`, this is invisible to:
- the checks UI (a green ✓ identical to a real pass)
- **branch protection** — `claude-review` as a required status check is satisfied by a run that never executed
- any automation gating on check conclusions (e.g. Dependabot auto-merge)
The only way to distinguish the two today is to open the run log and grep for `Exiting due to workflow validation skip`, or notice the ~10x duration difference. We were caught by this in practice: a workflow-fix PR showed green, and it took reading the log to establish that the check had done nothing.
## Suggested fix
Report the skip as **`neutral`** (GitHub's conclusion for "ran but intentionally reached no verdict") rather than `success`.
This preserves everything #1343 asked for — no spurious red ✗, both paths consistent, no "something is broken" impression — while keeping the outcome distinguishable from a real pass. Concretely, `neutral`:
- does **not** satisfy a required status check, so a workflow-editing PR can no longer merge on the strength of a review that never ran
- renders distinctly in the checks UI
- is already the conventional signal for "skipped by design"
If `neutral` is considered too disruptive for existing users relying on the green check, a lower-impact alternative would be to keep `success` but set a distinguishable check **output/summary** (e.g. title "Skipped — workflow modified") so tooling has something to key on without reading raw logs.
## Reproduction
1. Add `claude-code-action@v1` to a repo and merge it.
2. Open a PR that modifies that workflow file (change `--max-turns`, say).
3. Observe: check reports `success` in seconds; the log contains `Exiting due to workflow validation skip`; no review was performed.
4. Make `claude-review` a required status check and observe that the PR is mergeable on the strength of that skip.
## Workaround for others hitting this
Until upstream changes, a separate workflow filtered on the invoking workflow's path can post a notice on affected PRs. Worth noting the constraint that makes this awkward: the action cannot be fixed or tested from a PR that edits it, because that PR self-skips — the change has to land on the default branch first and be exercised on an unrelated PR.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.