microsoft / microsoft/hve-core
feat: detect silent soft-fail on security-relevant CI jobs
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 301
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 92
Description
### Issue Description
Build a validator that flags any security-relevant CI job in `.github/workflows/pr-validation.yml` whose `soft-fail` (or equivalent non-blocking) toggle is set to `true`, unless explicitly allow-listed with an inline comment justifying it.
Security-relevant jobs to cover include: `dependency-pinning-check`, `workflow-permissions-check`, `workflow-runner-check`, `dangerous-workflow-check`, `action-version-consistency-scan`, `npm-audit`, `codeql`, and any future job calling a `scripts/security/*.ps1` validator.
The check should:
* Parse job `with:` blocks for `soft-fail`/`*-soft-fail` inputs
* Fail if any security-relevant job sets it to `true` without an adjacent comment naming the reason (mirroring the existing `poutine-soft-fail: true # Poutine runs advisory; homegrown template-injection check is the hard gate` pattern)
* Pass if the toggle is `false`, or `true` with a justifying comment
Wire into `pr-validation.yml` and its `pr-validation-success` gate.
### Additional Context
**Why this matters:** A future PR could quietly flip a security job's soft-fail toggle to `true`, silently making it advisory instead of blocking, and nothing today would catch it. Current audit confirmed no such case exists yet, but there is no guardrail preventing one.
**Parent:** #2703
Contributor guide
Assessment
This issue has not been assessed yet.