microsoft / microsoft/hve-core

feat: detect silent soft-fail on security-relevant CI jobs

Open
#2,705 1 comment 1 reaction 1 assignee Claimed by @chalew101 View on GitHub
Dominant language
Python
Stars
1.5k
Forks
301
Avg merge
3d 3h
Merged PRs (30d)
92

Description

### Issue Description

Build a validator that flags any security-relevant CI job in `.github/workflows/pr-validation.yml` whose `soft-fail` (or equivalent non-blocking) toggle is set to `true`, unless explicitly allow-listed with an inline comment justifying it.

Security-relevant jobs to cover include: `dependency-pinning-check`, `workflow-permissions-check`, `workflow-runner-check`, `dangerous-workflow-check`, `action-version-consistency-scan`, `npm-audit`, `codeql`, and any future job calling a `scripts/security/*.ps1` validator.

The check should:

* Parse job `with:` blocks for `soft-fail`/`*-soft-fail` inputs
* Fail if any security-relevant job sets it to `true` without an adjacent comment naming the reason (mirroring the existing `poutine-soft-fail: true # Poutine runs advisory; homegrown template-injection check is the hard gate` pattern)
* Pass if the toggle is `false`, or `true` with a justifying comment

Wire into `pr-validation.yml` and its `pr-validation-success` gate.

### Additional Context

**Why this matters:** A future PR could quietly flip a security job's soft-fail toggle to `true`, silently making it advisory instead of blocking, and nothing today would catch it. Current audit confirmed no such case exists yet, but there is no guardrail preventing one.

**Parent:** #2703

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.