OpenZeppelin / OpenZeppelin/compact-contracts

dev: scope turbo compile outputs per module (fix 142 GB cache + 0-byte key races)

Open
#675 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
TypeScript
Stars
55
Forks
29
Avg merge
5d 7h
Merged PRs (30d)
25

Description

Problem

Every compile:* task in turbo.json sets outputs: ["artifacts/**/"] — the
whole shared tree, not its own slice. Two consequences:

  • Cache bloat: each entry is a ~1 GB snapshot of all artifacts, re-saved per
    input hash per task. .turbo/cache hit 142 GB (765 entries).
  • 0-byte key races: compile runs tasks in parallel over the same dir. A
    cache-hit task restores its full-tree snapshot on top of files a concurrent
    cache-miss task is still writing, so a stale/truncated *.verifier overwrites
    a fresh one. Observed: MockShieldedTreasury/keys/_deposit.verifier flipping
    0 → 2119 → 0 bytes → treasury live suite fails in beforeAll → 24 tests
    silently skipped.

Fix

Give each task its own output subtree so no two tasks share a path:

  1. Add --hierarchical to the compact-compiler calls (already used by build)
    → artifacts land under artifacts/<subdir>/<Contract>/.
  2. Scope each task's outputs to its subtree, e.g. compile:multisig
    ["artifacts/multisig/**"].
  3. Update artifact path refs (harness, #test-utils, deployer key loading).
  4. Force-recompile, run full + live suites.

Result: no shared paths → no restore-over-write races; per-module cache slices →
cache shrinks ~1–2 orders of magnitude.

Done as interim mitigation

  • scripts/test-live.ts hard-fails before the stack starts if compile left any
    0-byte *.verifier/*.prover.
  • One-time rm -rf .turbo/cache.

Stops the bleeding; does not remove the shared-path race — that is this issue.

Upstream follow-up (separate)

@openzeppelin/compact-cli should write keys atomically (temp + rename) and
handle SIGINT/SIGTERM, so an interrupted compile can't leave a truncated key.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in turbo.json by comparing the compile:* output definitions with the existing hierarchical build configuration, then trace artifact references in the harness, #test-utils, and deployer key loading. Force-recompile and run the full and live suites; done means each compile task has a separate artifact subtree, references still resolve, and the shared-path cache race is removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
build-system, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.