stacklok / stacklok/mecatl

ci.yml: path-scope the SDK and macOS job groups to cut Actions cost

Open
#1,356 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ci enhancement
Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Summary

ci.yml fans out its full job set on almost every push and PR. The only
change-based gate is a single docs_only boolean. A one-line change to any
non-docs file runs all ~25 jobs, including the 3 macOS jobs and the 8-leg
SDK/browser matrix, even when no SDK or macOS-relevant code changed.

Scope the SDK job group and the native/macOS job group to the paths they
actually exercise, driven by new changes outputs and per-job if:
conditions. Keep the current required-check design intact.

Cost context

Actions billing over a recent 7-day window shows macOS 3-core runners
consuming roughly 30% of total spend from about 4% of total minutes — macOS
3-core bills at roughly 10x the Linux 2-core rate, so a macOS job that runs
when it did not need to is the most expensive waste in the pipeline. The repo
uses no 4-core, 8-core, or large runners. Every macOS minute comes from
ci.yml.

Root cause

ci.yml has a changes job (ci.yml:~40) that emits one output, docs_only.
Every job gates on if: needs.changes.outputs.docs_only != 'true'. There is no
finer scoping, so SDK-only, website-only, helm-only, and user-docs-only changes
still start the macOS jobs and the SDK matrix.

Constraints the fix MUST respect

  1. Do NOT add workflow-level paths: filters. The existing design note at
    ci.yml:55 is explicit: required checks must exist on every PR. A required
    context that never runs blocks the PR. Keep gating with per-job if: so a
    skipped job still reports a completed check.
  2. Preserve the always() aggregator jobs (test at ci.yml:711, lint at
    ci.yml:915). They emit the single stable required context regardless of what
    ran. Any newly gated job that feeds an aggregator must keep the aggregator
    green when the job is skipped.
  3. Keep the base-ref classifier security property. docs-only-changes.sh is
    read from the base SHA (ci.yml changes step), so a PR cannot weaken its own
    gate. Any new classification MUST read its allowlist from the base ref the
    same way. This rules out dorny/paths-filter in its default form, which
    reads the filter config from the PR head.

Proposed change

Extend the changes job to emit two more booleans next to docs_only, then
gate the SDK and native job groups on them.

New outputs (names indicative):

  • sdk_touched: paths under sdk/typescript/**, contracts/**,
    contracts/gen/**, plus the files whose change regenerates contracts
    (go.mod, go.sum, Taskfile.yml, .github/workflows/ci.yml).
  • native_touched: paths that affect a built binary or its macOS behavior:
    cmd/**, internal/**, engine/**, authn/**, provider/**, go.mod,
    go.sum, .goreleaser.yaml, Taskfile.yml, .github/workflows/ci.yml.

Implement these by extending the trusted classifier (a multi-output variant of
docs-only-changes.sh, still NUL-delimited, still read from the base SHA, still
never evaluating a path as shell), or by a second trusted script. Emit each
boolean on $GITHUB_OUTPUT.

Job gates (change the if: only; leave steps unchanged):

Job ci.yml line Runner New gate (in addition to docs_only != 'true')
sdk 112 ubuntu-24.04 (x2 node) sdk_touched == 'true'
sdk-integration 169 ubuntu-24.04 sdk_touched == 'true'
sdk-browser 220 ubuntu-24.04 (x3 browser) sdk_touched == 'true'
sdk-macos-spawn 290 macos-14 sdk_touched == 'true'
smoke-darwin-arm64 476 macos-14 native_touched == 'true'
managed-temp-macos 515 macos-14 native_touched == 'true'

Also add && github.event.pull_request.draft == false to the three macOS jobs
so a draft PR does not spend macOS minutes before it is ready. The race tests
already use that draft guard; the macOS and SDK jobs do not.

Always run the full set on push: main regardless of the new outputs, so main
keeps end-to-end coverage and the shared Go module cache still populates from a
complete build.

Correctness risk and mitigation

Path-gated end-to-end jobs can miss a cross-cutting regression. The SDK e2e
builds mecated from the same checkout, so a pure server change that alters
runtime behavior without touching contracts/gen could regress the SDK on a PR
that skips the SDK jobs.

Mitigations:

  • Include the server and engine Go packages in sdk_touched (the paths above
    cover contracts/** and the generation inputs; widen to the specific server
    packages the SDK e2e depends on if needed). Prefer a slightly wider filter
    over a missed regression.
  • push: main always runs the full suite, so any gap is caught at merge, not
    released.
  • Optional: honor a label such as ci-full to force the complete set on a PR
    when a change is known to be cross-cutting.

Expected impact

Targets the macOS spend line and the SDK/browser matrix. SDK-only PRs
stop running the native macOS smoke and the macOS temp test. Non-SDK PRs stop
running the two SDK macOS jobs and the six-leg SDK/browser matrix. Exact savings
depend on the PR mix; the macOS spend is the largest single lever.

Acceptance criteria

  • changes emits sdk_touched and native_touched, computed from the base
    ref, with a classifier that never evaluates a path as shell.
  • The six jobs above gate on the new outputs; the three macOS jobs also gate
    on draft == false.
  • push: main still runs the full job set.
  • The test and lint aggregators stay green when the gated jobs skip; the
    set of required contexts on a PR is unchanged.
  • Verified on three PR shapes: SDK-only (no macOS, no core Go), docs-only
    (unchanged behavior), and a server Go change (native jobs run, SDK jobs run
    per the chosen sdk_touched breadth).

Out of scope

  • e2e-live.yml runs on every PR with no path filter and deserves its own
    scoping change (label-gate or path-gate), tracked separately.
  • No runner-size downgrades. Every job stays on its current runner; this issue
    changes only WHEN jobs run.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the changes job and gated jobs in .github/workflows/ci.yml, then inspect docs-only-changes.sh and its base-ref handling. Verify the new outputs, per-job conditions, draft guards, push: main behavior, and always() aggregators against the three stated PR shapes; done means the acceptance criteria pass without changing required contexts.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, go, shell
Domain
build-system, ci-cd, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.