IntersectMBO / IntersectMBO/mithril
Persist the SNARK test caches across CI runs
- Dominant language
- Rust
- Stars
- 154
- Forks
- 51
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 38
Description
## Why
Neither the keygen cache nor the SRS is persisted across CI runs (the `toolchain-and-cache` action only saves dependency artifacts through `Swatinem/rust-cache`), so every nightly and every slow PR leg starts cold.
## What
Add an `actions/cache` layer for `target/mithril-circuit-test-cache` with a three-layer invalidation design: content-keyed directories for semantic staleness, a versions-scoped cache key for serialization format drift, and self-healing readers as backstop.
## How
Paths are relative to `mithril-stm/src/` unless they start with `.github/`.
- [ ] Add an `actions/cache/restore` step for `target/mithril-circuit-test-cache` in the test job of `.github/workflows/test-rust.yml` (next to the `toolchain-and-cache` composite step), with an exact key folding a manual generation token and a hash of the `midnight-*` entries of `Cargo.lock`
- [ ] Scope every `restore-keys` prefix to that versions hash so a `midnight-proofs` upgrade (serialization format drift) can never restore stale bytes
- [ ] Save with `actions/cache/save` only from the runs that execute the full slow suite (the `include-slow-tests: true` invocation from `nightly-dispatcher.yml`); PR runs stay restore-only
- [ ] Make the readers self-healing: on deserialization failure, `KeyProvider::cache_state` and `read_optional` (`circuits/key_provider.rs`) and `TrustedSetupProvider::get_trusted_setup_parameters` (`circuits/trusted_setup.rs`) should delete the offending file and regenerate instead of failing the test
- [ ] Prune fingerprint directories not touched by the current run before saving, so retired fingerprints do not accumulate in the archive
- [ ] Make one recursive verifying-key cache canonical for both the circuit tests and `IvcSnarkProverSetup::build_for_test_degree`, with the prover path storing only the additional proving key: today `proof_system/ivc_halo2_snark/prover_setup.rs` and `circuits/halo2_ivc/tests/common/generators/setup.rs` hold the same key under two protocols, because `KeyProvider::verification_key` treats a missing proving key as a miss and regenerates the pair
- [ ] Assess the gain (expected: removes the cold start from warm CI runs) and report it in the issue
Depends on #3433.
Contributor guide
Research direction
First check the dependency on #3433, then read the test job in .github/workflows/test-rust.yml and the cache readers in circuits/key_provider.rs and circuits/trusted_setup.rs. Compare the duplicate cache paths in proof_system/ivc_halo2_snark/prover_setup.rs and circuits/halo2_ivc/tests/common/generators/setup.rs. Done means warm CI restores safely, stale entries self-heal, only full slow-suite runs save pruned caches, and the issue reports the measured gain.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, rust
- Domain
- build-system, ci-cd, cryptography, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100