CI hardening: provider API keys exposed on pull_request (evals.yml) + third-party actions pinned to mutable tags
- Dominant language
- TypeScript
- Stars
- 133k
- Forks
- 19.9k
- Avg merge
- 18h 46m
- Merged PRs (30d)
- 26
Description
## Summary
Two CI/CD supply-chain hardening gaps, found during a security review of v1.57.7.0.
### 1. Provider API keys are available to `pull_request`-triggered runs
`.github/workflows/evals.yml` is triggered `on: pull_request` (line 3) and runs the test matrix on PR-authored code with real provider keys in env:
```yaml
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} # :142
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} # :143
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }} # :144
```
GitHub does not pass secrets to **fork** PRs unless the repo opts in / a maintainer approves the run — so this is safe **only** if that setting is locked down. If "send secrets to workflows from fork pull requests" is enabled (or for any branch PR from a write-access contributor), a PR that edits a matrix test/build file executes arbitrary code with all three keys in the environment → key exfiltration. (`evals-periodic.yml` uses the same keys but is schedule-triggered, so it isn't fork-reachable.)
**Fix:** confirm fork-PR secret passing is disabled and rely on maintainer "approve and run"; better, split the secret-bearing eval runs into a gated stage (label-triggered, merge-queue, or `workflow_dispatch`) so the fork-facing PR check runs without provider keys.
### 2. Third-party actions are pinned to mutable tags, not commit SHAs
Every action reference uses a moving tag rather than a pinned SHA, e.g.:
- `docker/login-action@v3`, `docker/build-push-action@v6` — `evals.yml:30,49`; `evals-periodic.yml:30,49`; `ci-image.yml`
- `oven-sh/setup-bun@v1` / `@v2` — `windows-free-tests.yml`, `windows-setup-e2e.yml`, `skill-docs.yml`, `version-gate.yml`
- `actions/checkout@v4`, `actions/upload-artifact@v4`, `actions/download-artifact@v4` — throughout
If any tag is force-moved (compromised maintainer / hijacked org), the next run executes attacker code — including in the secret-bearing `evals` job (#1 above) and the `pull_request_target` `pr-title-sync` job (which holds a write token). `docker/build-push-action` runs in jobs with `packages: write` and pushes `ghcr.io//ci`, so a compromise there poisons the CI base image consumed by later eval runs.
**Fix:** pin actions to full 40-char commit SHAs (with a `# vX.Y.Z` trailing comment) and let Dependabot/Renovate bump them; pin `bun-version` to an explicit version rather than `latest`.
---
*Related and already filed: #1706 (avoid piping the remote Bun installer straight into the CI shell). The GitHub `pr-title-sync.yml` workflow, by contrast, is correctly hardened — untrusted PR fields go through `env:` and it checks out the base repo only.*
Contributor guide
Assessment
This issue has not been assessed yet.