Discussion: how do we prevent loose CI dependencies from being merged?

Context

The three loose-dependency audits are filed:

• OpenHands/software-agent-sdk#4304 — software-agent-sdk
• OpenHands/OpenHands#16161
• OpenHands/enterprise#96

Those issues are the inventory of loose ends (unpinned Actions, floating installer scripts, unbounded dependency ranges, unpinned Docker base images, etc.) — the class of finding where an unchanged commit can flip from green to red, or resolve to code nobody reviewed, with zero code changes on our side.

This issue is about how we stop new ones from getting merged, and keep the fixed ones fixed — before any of that inventory gets closed out repo by repo. Opening this for discussion first rather than just landing a PR, since it's an org-wide policy/tooling decision, not a local fix.

Proposed approach

Layered, because "pin everything" and "prevent new looseness" and "keep pins fresh" are three different problems needing three different tools:

New rule adopted for this table: if we can't find a credible OSS project actually using a tool, that's a reason to question whether we should either — so every row below was re-checked against real GitHub code search (not just docs/blog posts) after an initial pass turned up thinner evidence than it should have. Everything below now has genuine, verifiable adopters; nothing survived that didn't.

Full tool research (why these over the alternatives, e.g. Dependabot vs Renovate, Socket.dev as an optional paid layer, etc.) is in the conversation that produced this issue — happy to write that up as a wiki page / doc if useful rather than re-litigating here.

Where these solutions are already used

Pulled this together specifically to sanity-check credibility — is this a coherent, independently-converged-upon set of practices, or just a list of tools that sounded good? A few honest data points, including the ones that don't flatter the pitch:

• Homebrew-core is the closest structural match to us (huge PR-review-driven community-contribution volume, not a small core team) and has already independently converged on zizmor + explicit pull_request_target avoidance + Sigstore build attestations + a dependency cooldown — nobody there was reading our issue, they landed in the same place from the same problem. Their own docs put it plainly: "There is always a human in the loop between an upstream release and that release reaching Homebrew users." (docs.brew.sh/Homebrew-Security-and-Supply-Chain)
• PyPI's own Warehouse repo — the registry underpinning Trusted Publishing and PEP 740 attestations for the entire Python ecosystem — runs zizmor's full, unscoped ruleset as a required gate on every push/PR, not a "phase 2" nice-to-have. Also SHA-pins every Action and sets persist-credentials: false (the exact artipacked finding our own pilot deliberately deferred as out-of-scope).
• curl's Daniel Stenberg has a named, public philosophy — "verification over trust" — and explicitly recommends zizmor for exactly this class of problem (InfoQ, May 2026).
• The March 2025 tj-actions/changed-files attack (attacker rewrote ~all version tags on an action used by ~23,000 repos to exfiltrate CI secrets) is the industry-wide proof-point that tag-pinning isn't real pinning — only a SHA is immutable. crates.io's own repo still uses that exact action today, just SHA-pinned rather than removed — pinning (not avoidance) is treated as sufficient mitigation even for a previously-compromised action.
• Kubernetes has Cosign-signed every release artifact since v1.24 (April 2022), verifiable against the public Rekor transparency log — the same Sigstore/SLSA stack in Layer 6, already validated at massive scale.
• Honest counter-evidence, so this doesn't read as cherry-picked: sigstore/cosign — the tool that is supply-chain security — scores 0/10 on OpenSSF Scorecard's Token-Permissions check. actions/checkout, the single most-used Action in the entire ecosystem and authored by GitHub itself, does not SHA-pin any of its own Action references. Nobody in this research is uniformly perfect; the pattern is picking the 2–3 dimensions that matter most for your own threat model and being genuinely rigorous there, which is what the layered/phased structure above is trying to do rather than chasing a uniform 10/10.
• One data point specific to our own audit's mcp-server-fetch finding: the Model Context Protocol's own official security best-practices doc (modelcontextprotocol.io) recommends, from the protocol authors themselves, to "pin the version of the MCP server and notify the user if code/composition changes after install." Worth knowing this isn't just our opinion — it's the spec authors' own stated mitigation for a currently-unpatched class of MCP RCE.
• On "don't adopt a tool nobody credible uses": a first pass at the table's "Used by" column had a few cells reading "not confirmed" — worth being honest that the first round of research wasn't thorough enough, not that the adoption doesn't exist. A second, code-search-driven pass (real GitHub code search across workflows/configs, not just docs/blog posts) found genuine adopters for every tool, some stronger than expected — minimumReleaseAge in particular turned out to be a default in pnpm v11 now, not a niche opt-in, and harden-runner turned out to be running in Node.js core itself. Nothing in the table below is there without a verifiable adopter.

What a pilot on this repo (software-agent-sdk) surfaced

Before proposing this org-wide, I built a throwaway branch here to sanity-check that Layers 1–3 actually work on a real repo rather than just sounding good in the abstract. Not pushed — this is a design discussion first, per the plan. Findings worth knowing before we commit to an approach:

• pinact + zizmor together caught something pinact alone would have missed: the SHA already pinned for issue-duplicate-checker in 3 workflows (pre-existing, unrelated to this pilot) turned out to be an orphaned pre-squash-merge commit — reachable only via a stale PR-head ref in OpenHands/extensions, not any branch or tag. GitHub isn't obligated to retain that forever. This is exactly the "impostor-commit" class of finding zizmor is built to catch and pinact's simpler SHA-presence check can't.
• Found a live cache-poisoning path: astral-sh/setup-uv's default caching was enabled in 3 release-publish workflows (pypi-release, release-binaries, server) — a poisoned cache could ride along into a published artifact. One-line fix (enable-cache: false), but only zizmor's cache-poisoning audit surfaces it.
• Found something more serious that needs its own review, not a mechanical fix: 5 workflows use pull_request_target, and at least one (integration-runner.yml) checks out untrusted fork PR code under that trigger — the exact bug class behind the March 2026 trivy-action→LiteLLM PyPI backdoor. This deserves a dedicated, careful look at each workflow's gating logic before we touch it. Flagging it here since it's arguably more urgent than the pinning work, even though it's outside this issue's original scope.
• A required CI gate has to be scoped to what's already clean, or it breaks on day one. The full zizmor/hadolint rulesets have real, pre-existing findings (Actions permissions, credential persistence, Dockerfile package-pinning) that aren't fixable as a quick pass. The pilot's gate only blocks on the subset that's already green today (so it only ever catches new regressions); everything else runs informationally so the backlog stays visible instead of getting silently swept under the rug or blocking every PR on day one.
• Don't duplicate what's already there: this repo already has a hand-built release-boundary OSV/dependency-confusion check (security-scan.yml + check_dependency_diff.py) that's more tailored than the generic dependency-review-action I'd originally planned to bolt on. Worth checking each repo for similar existing infra before assuming a generic tool is the gap-filler.
• Swapping Dependabot for Renovate means updating the couple of places that special-case dependabot[bot] as a trusted automation identity (PR-description check, approval-drift security scan) to also trust renovate[bot].

None of the above is committed code — it's what a few hours of actually trying this surfaced, offered as evidence for the discussion rather than a fait accompli.

Open questions for this discussion

1. Scope for v1: Layers 1–3 only (pin, keep-fresh, block-regressions), or do we want Layer 4/5 (Scorecard, Allstar, harden-runner) in the same pass?
2. Rollout order: pilot fully on software-agent-sdk → OpenHands → enterprise, or parallelize across repos once the approach is agreed?
3. The pull_request_target findings: separate issue/owner, or fold into this effort?
4. Org-level policy (Layer 0): who owns getting org-owner buy-in for the SHA-pinning-required toggle, and should we sequence it after Layers 1–3 land everywhere (so it doesn't break workflows still on tags)?
5. Budget/tooling constraints — StepSecurity and Socket.dev both have free tiers but paid ceilings; confirming free/OSS-only is fine for v1 unless someone wants more.

cc anyone who owns CI/release tooling on the other two repos — this affects all three.

Read the three linked audit issues and the pilot findings here, then inspect integration-runner.yml and the named release workflows. Compare the proposed layers and their existing adopters, including pinact, zizmor, Renovate and Scorecard. Done means an agreed, scoped organization-wide policy and tooling plan rather than an implementation patch.