0xMiden / 0xMiden/node

Reduce / remove MMR clone in apply block

Abierto
#2,436 1 comentario 0 reacciones 1 asignado Reclamado por @sergerad Ver en GitHub
blocked
Lenguaje dominante
Rust
Estrellas
104
Forks
138
Merge medio
1 d 13 h
PR fusionados (30 d)
56

Descripción

## What the clone actually copies

`Blockchain` is a thin wrapper around `miden_crypto::Mmr`, which is `{ forest: Forest, nodes: Vec }` — a single contiguous vector holding *every* node of the mountain range in postorder. `Clone` is a full memcpy of that vector. An MMR with N leaves has ~2N − popcount(N) nodes, and each `Word` is 32 bytes, so the chain costs **~64 bytes per block**, all deep-copied on every `apply_block`:

| Chain length | Nodes vec size | Clone time (memcpy ~10–30 GB/s) |
|---|---|---|
| 100k blocks | ~6.4 MB | < 1 ms |
| 1M blocks (~2 months at 5s blocks) | ~64 MB | ~3–8 ms |
| 6.3M blocks (~1 year) | ~400 MB | ~20–50 ms |
| 5 years | ~2 GB | ~100–300 ms |

So today it's negligible, but it grows linearly forever and sits on the apply path (inside `run_on_pool`), directly adding to block-apply latency and snapshot-publication delay. The memory story is worse than the CPU story: the writer holds one copy, the currently published snapshot holds another, and every older snapshot pinned by an in-flight reader holds its own — at year-one scale that's 400 MB × (2 + in-flight readers), plus a 400 MB allocation churned every block.

## Why we can't just shrink it

Snapshot readers call `blockchain().open(...)`, `open_at(...)`, and `partial_mmr_from_blocks(...)` (`view/block.rs:37`, `view/sync.rs:129`, `view/batch_inputs.rs:93`) to build MMR proofs for arbitrary historical blocks. So a peaks-only (`MmrPeaks`/`PartialMmr`) snapshot doesn't work — readers genuinely need the full node set.

Cheap tricks also don't work here:
- `Arc` + `Arc::make_mut` degenerates to clone-per-block, because the currently published snapshot always holds the previous Arc when the writer pushes the next leaf — the Arc is never unique.
- The node can't restructure storage itself: `Mmr.nodes` is private; `Blockchain` only exposes `from_mmr_unchecked(Mmr)`.

## What actually fixes it

The MMR has the perfect property for this: **`push` only ever appends to `nodes` — existing entries are never mutated** (that's the point of the postorder representation). A snapshot at forest F is literally just a prefix of the vector plus the checkpoint value, and `Blockchain` already has checkpoint-parameterized APIs (`open_at`, etc.).

So the right fix is an upstream snapshot API in miden-crypto, mirroring what the nullifier/account trees already give you via `.reader()`:

1. **Chunked shared storage**: store nodes as segments (`Vec>` or similar). `Mmr::reader()` returns `(cloned spine of Arcs, Forest)` — O(N/CHUNK) pointer copies, effectively O(1), zero node data copied. The writer appends into the last unshared chunk. Readers only index below their checkpoint, which is immutable by construction.
2. **Or even simpler**: an append-only shared buffer (stable-address segmented vec with an atomic length) wrapped in one `Arc`; snapshot = `Arc` clone + `Forest`. Same immutable-prefix argument makes it sound without locks.

Either way `worker.rs:398` becomes `self.blockchain.reader()` and the snapshot line stops being O(chain length).

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.