MMR migration: switch from hashed peaks to a rooted Merkle frontier
- Dominant language
- Rust
- Stars
- 772
- Forks
- 352
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 93
Description
This issue proposes a narrower version of the current MMR.
This direction also came up in the discussion on [0xMiden/miden-vm#3512](https://github.com/0xMiden/miden-vm/issues/3512#issuecomment-4274263292).
The idea starts from two simple facts about append-only Merkle trees:
- If you know the Merkle path for a leaf position, you can swap in a new value at that same position and recompute the root without touching the rest of the tree.
- In an append-only tree, once all leaves after position `i` are still empty, the path to the next leaf can be stored in a compact way.
That gives a compact frontier:
- `len` is the next leaf index,
- `peaks` are the full left subtrees on the path to `len`,
- empty subtrees on the right are implied,
- the public commitment is the Merkle root of that path.
With that model, the change is simple:
- keep the append-only shape,
- keep the `len + peaks` frontier,
- stop using `hash(peaks)` as the public commitment,
- use one normal Merkle root instead,
- verify reads with normal Merkle proofs.
This puts `len` inside the commitment instead of leaving it as side data that every caller must handle with care.
# why
- The current design commits to `hash(peaks)`, not to the full append-only tree shape.
- That makes `len` a special case.
- The rooted form is easier to reason about because the root already fixes the shape.
- Reads become plain Merkle verification.
- Appends still use the same cheap binary-carry update.
# model
For `len = 13 = 1101₂`, the path to the next leaf looks like this:
```mermaid
flowchart BT
E0["empty leaf at index 13"] --> N0
P1["peak(1)"] --> N0
N0 --> N1
E2["empty(2)"] --> N1
N1 --> N2
P4["peak(4)"] --> N2
N2 --> N3
P8["peak(8)"] --> N3
N3 --> R["root"]
```
Each bit of `len` tells you which side is full:
- bit `1`: a full subtree is on the left, so consume one peak,
- bit `0`: the sibling on the right is an empty subtree.
# algorithm
`append(leaf)` is still the same binary carry now used by `Mmr::add()`.
- Start with the new leaf as a subtree of size `1`.
- While the low bit of `len` is `1`, pop the last peak and merge it on the left.
- Push the final merged node as the new last peak.
- Increase `len`.
- Recompute or cache the single Merkle root for the new frontier.
`root()` is computed by folding the path to `len`.
```rust
pub struct MerkleFrontier {
len: usize,
peaks: Vec, // same order as MmrPeaks: largest -> smallest
}
impl MerkleFrontier {
pub fn root(&self) -> Word {
if self.len == 0 {
return empty_subtree_root(0);
}
let mut bits = self.len;
let mut level = 0u8;
let mut peak_idx = self.peaks.len();
let mut acc = empty_subtree_root(0);
while bits != 0 {
if bits & 1 == 1 {
peak_idx -= 1;
acc = merge(self.peaks[peak_idx], acc);
} else {
acc = merge(acc, empty_subtree_root(level));
}
bits >>= 1;
level += 1;
}
acc
}
pub fn append(&mut self, mut node: Word) {
let mut n = self.len;
while n & 1 == 1 {
let left = self.peaks.pop().expect("peak must exist");
node = merge(left, node);
n >>= 1;
}
self.peaks.push(node);
self.len += 1;
}
}
```
If a protocol wants a fixed tree depth, keep folding with empty roots after the last set bit.
# API shape
The public API can look like a normal Merkle API:
| Today | Proposed |
|---|---|
| `MmrPeaks::hash_peaks()` | `MerkleFrontier::root()` |
| `MmrPeaks::verify(value, MmrProof)` | `MerklePath::verify(index, value, &root)` |
| `MmrProof` | plain `MerkleProof` or `(index, value, MerklePath)` |
| `MmrPeaks` as public state | `MerkleFrontier` as public state |
| `Forest` as public version tag | `len` as public version tag |
The frontier can still stay peak-based inside. The real change is at the API edge and in the commitment.
# migration plan
- If a consumer only stores a commitment, switch from `hash(peaks)` to `root`.
- If a consumer verifies reads, switch from `MmrProof` to standard `MerklePath` verification.
- If a consumer appends, store `len + peaks` behind a small wrapper and keep the current carry logic.
- If a consumer produces proofs, keep the current `open()` logic inside first, but change the API edge to return plain Merkle proofs against `root`.
- If a consumer syncs partial state, keep the existing delta logic first and change the public commitment first.
- If a consumer tracks a subset of leaves, base the next type on `PartialMmr`, but make the public anchor a single root, not `hash(peaks)`.
One clean rollout is:
1. Add `MerkleFrontier` next to the current MMR types.
2. Add `root()` and plain Merkle proof verification.
3. Mark `hash_peaks()` as legacy.
4. Move public callers to the rooted API.
5. Remove peak-hash commitments after downstream code is off them.
# facade sketch
A small facade can bridge the two APIs:
```rust
pub struct MerkleFrontierFacade {
inner: MmrPeaks,
}
impl MerkleFrontierFacade {
pub fn num_leaves(&self) -> usize {
self.inner.num_leaves()
}
pub fn root(&self) -> Word {
MerkleFrontier {
len: self.inner.num_leaves(),
peaks: self.inner.peaks().to_vec(),
}
.root()
}
pub fn into_legacy_peaks(self) -> MmrPeaks {
self.inner
}
}
```
That gives downstream code a clear path:
- first change the commitment type,
- then change proof production and verification,
- then drop the peak-hash API.
# code pointers in `0xMiden/crypto`
Pointers below use `0xMiden/crypto@68ac2cf0b8473a63998b38149f8d599ffe3179b8`.
- Current append carry logic in full MMR: [`miden-crypto/src/merkle/mmr/full.rs#L153-L185`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/full.rs#L153-L185)
- Current proof opening and peak extraction: [`miden-crypto/src/merkle/mmr/full.rs#L116-L215`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/full.rs#L116-L215)
- Current peak-hash commitment and peak-local verification: [`miden-crypto/src/merkle/mmr/peaks.rs#L120-L175`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/peaks.rs#L120-L175)
- Current proof types: [`miden-crypto/src/merkle/mmr/proof.rs#L10-L163`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/proof.rs#L10-L163)
- Current path builder inside the full MMR: [`miden-crypto/src/merkle/mmr/full.rs#L315-L378`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/full.rs#L315-L378)
- Current partial-state append and tracking logic: [`miden-crypto/src/merkle/mmr/partial.rs#L304-L445`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/partial.rs#L304-L445)
- Current partial-state delta apply logic: [`miden-crypto/src/merkle/mmr/partial.rs#L514-L643`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/partial.rs#L514-L643)
- Bit-level forest helpers that already encode the frontier shape: [`miden-crypto/src/merkle/mmr/forest.rs#L12-L110`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/forest.rs#L12-L110) and [`miden-crypto/src/merkle/mmr/forest.rs#L308-L445`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/forest.rs#L308-L445)
- Standard Merkle proof verification to reuse: [`miden-crypto/src/merkle/path.rs#L58-L86`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/path.rs#L58-L86)
- Empty subtree roots to reuse for the frontier path: [`miden-crypto/src/merkle/empty_roots.rs#L8-L33`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/empty_roots.rs#L8-L33)
- Store integration that may want a new rooted adapter later: [`miden-crypto/src/merkle/store/mod.rs#L533-L537`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/store/mod.rs#L533-L537)
# scope
This does not ask for a new general tree.
- It keeps the same append-only frontier model.
- It keeps the same cheap updates.
- It only narrows the public model to the rooted frontier described above.
Contributor guide
Assessment
This issue has not been assessed yet.