ci: darwin release builds are uncached — port rivet sccache/cache-mount Dockerfile pattern
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 4.6k
- Forks
- 251
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 34
Description
The "Build darwin binaries" jobs are the critical path for Publish npm in publish.yaml (~10.5 min per release) and rebuild ~388 crates from scratch every run.
Current state
docker/build/darwin.DockerfileusesFROM ghcr.io/rivet-dev/rivet/builder-base-osxcross:0e33ceb98— the same prebaked base rivet uses (osxcross + MacOSX 11.3 SDK + both apple Rust targets + sccache preinstalled) — but uses none of its caching affordances:- re-installs the rust toolchain/targets the base already has
- no
RUSTC_WRAPPER=sccache, no cache backend, no BuildKit cache mounts - plain
docker buildwith no--cache-from/--cache-to
- Measured on the 0.2.5 stable run (28801821648): base image pull ~1 min, rustup ~1 min,
cargo build --releaseof 388 crates = 7m52s, every run, forever. - By contrast the linux sidecar/plugin jobs use Swatinem/rust-cache and finish in 1–5 min warm.
Fix: port the rivet consumer pattern
Rivet already solved this for the identical build in rivet-dev/rivet:
docker/build/darwin-{arm64,x64}.Dockerfile:RUSTC_WRAPPER=sccache+SCCACHE_WEBDAV_ENDPOINT=https://cache.depot.dev,--mount=type=secret,id=DEPOT_TOKEN,env=SCCACHE_WEBDAV_TOKENwith a graceful health-check fallback that disables sccache when the token/backend is unavailable, plus--mount=type=cachefor cargo registry/git/target.- Base image build/push lives in
docker/builder-base/osxcross.Dockerfile+scripts/docker-builder-base/build-push.sh.
Port notes for agentos:
- agentos builds on GitHub-hosted runners (not Depot builders), so BuildKit cache mounts alone will not persist across runs — the cross-run win is the sccache remote backend. Either:
- (a) add a
DEPOT_TOKENsecret and reuse rivet's Depot cache config verbatim (preferred: byte-for-byte port of a production pattern), or - (b) use sccache's native GHA backend (
SCCACHE_GHA_ENABLED=on).
- (a) add a
- Drop the redundant rustup toolchain/target install from
darwin.Dockerfile(already in the base). - Pass the secret through in the
build-sidecar-darwinjob in.github/workflows/publish.yaml(docker buildx build --secret id=DEPOT_TOKEN).
Expected: darwin job drops from ~10 min to ~2–3 min warm, cutting release npm-publish latency roughly in half.
Related smaller wins (same investigation, can be separate PRs)
Publish crates.iojob: give it its own concurrency group (or workflow) so a slow crates tail (crates.io new-crate rate-limit backoff can sleep for many minutes) doesn't queue the next release's npm path behind it (concurrency: publish-${{ github.ref }},cancel-in-progress: falsecaused the 0.2.6 dispatch to sit pending behind 0.2.5's crates job). Also surface the "rate limited, retrying at " sleep as a::notice::so the job doesn't look hung.- Release rust-cache eviction: linux jobs key caches by
<target>-<trigger>, so-releasecaches are only warmed by prior release runs and get LRU-evicted during quiet stretches while-previewstays warm; consider a shared key or a scheduled warm job. - WASM Commands (~8.6 min): cache
packages/runtime-core/commandskeyed onhashFiles('registry/native/**')and skipmakeon hit.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Compare docker/build/darwin.Dockerfile with rivet's docker/build/darwin-{arm64,x64}.Dockerfile, then inspect the build-sidecar-darwin job in .github/workflows/publish.yaml and the named builder-base files. Verify the resulting warm darwin release build uses the remote cache, avoids redundant rustup setup, and falls back gracefully when the cache secret is unavailable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, github-actions, rust
- Domain
- build-system, ci-cd, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100