Reduce / remove MMR clone in apply block
- 主要语言
- Rust
- 星标
- 104
- 派生
- 138
- 平均合并
- 1 天 13 小时
- 30 天内合并 PR
- 56
描述
## What the clone actually copies
`Blockchain` is a thin wrapper around `miden_crypto::Mmr`, which is `{ forest: Forest, nodes: Vec }` — a single contiguous vector holding *every* node of the mountain range in postorder. `Clone` is a full memcpy of that vector. An MMR with N leaves has ~2N − popcount(N) nodes, and each `Word` is 32 bytes, so the chain costs **~64 bytes per block**, all deep-copied on every `apply_block`:
| Chain length | Nodes vec size | Clone time (memcpy ~10–30 GB/s) |
|---|---|---|
| 100k blocks | ~6.4 MB | < 1 ms |
| 1M blocks (~2 months at 5s blocks) | ~64 MB | ~3–8 ms |
| 6.3M blocks (~1 year) | ~400 MB | ~20–50 ms |
| 5 years | ~2 GB | ~100–300 ms |
So today it's negligible, but it grows linearly forever and sits on the apply path (inside `run_on_pool`), directly adding to block-apply latency and snapshot-publication delay. The memory story is worse than the CPU story: the writer holds one copy, the currently published snapshot holds another, and every older snapshot pinned by an in-flight reader holds its own — at year-one scale that's 400 MB × (2 + in-flight readers), plus a 400 MB allocation churned every block.
## Why we can't just shrink it
Snapshot readers call `blockchain().open(...)`, `open_at(...)`, and `partial_mmr_from_blocks(...)` (`view/block.rs:37`, `view/sync.rs:129`, `view/batch_inputs.rs:93`) to build MMR proofs for arbitrary historical blocks. So a peaks-only (`MmrPeaks`/`PartialMmr`) snapshot doesn't work — readers genuinely need the full node set.
Cheap tricks also don't work here:
- `Arc` + `Arc::make_mut` degenerates to clone-per-block, because the currently published snapshot always holds the previous Arc when the writer pushes the next leaf — the Arc is never unique.
- The node can't restructure storage itself: `Mmr.nodes` is private; `Blockchain` only exposes `from_mmr_unchecked(Mmr)`.
## What actually fixes it
The MMR has the perfect property for this: **`push` only ever appends to `nodes` — existing entries are never mutated** (that's the point of the postorder representation). A snapshot at forest F is literally just a prefix of the vector plus the checkpoint value, and `Blockchain` already has checkpoint-parameterized APIs (`open_at`, etc.).
So the right fix is an upstream snapshot API in miden-crypto, mirroring what the nullifier/account trees already give you via `.reader()`:
1. **Chunked shared storage**: store nodes as segments (`Vec>` or similar). `Mmr::reader()` returns `(cloned spine of Arcs, Forest)` — O(N/CHUNK) pointer copies, effectively O(1), zero node data copied. The writer appends into the last unshared chunk. Readers only index below their checkpoint, which is immutable by construction.
2. **Or even simpler**: an append-only shared buffer (stable-address segmented vec with an atomic length) wrapped in one `Arc`; snapshot = `Arc` clone + `Forest`. Same immutable-prefix argument makes it sound without locks.
Either way `worker.rs:398` becomes `self.blockchain.reader()` and the snapshot line stops being O(chain length).
贡献指南
评估
这个 Issue 还没有评估数据。