0xMiden / 0xMiden/miden-vm

MMR migration: switch from hashed peaks to a rooted Merkle frontier

Open
#3,526 1 comment 1 reaction 0 assignees View on GitHub
corelib merkle
Dominant language
Rust
Stars
772
Forks
352
Avg merge
1d 12h
Merged PRs (30d)
93

Description

This issue proposes a narrower version of the current MMR.

This direction also came up in the discussion on [0xMiden/miden-vm#3512](https://github.com/0xMiden/miden-vm/issues/3512#issuecomment-4274263292).

The idea starts from two simple facts about append-only Merkle trees:

- If you know the Merkle path for a leaf position, you can swap in a new value at that same position and recompute the root without touching the rest of the tree.
- In an append-only tree, once all leaves after position `i` are still empty, the path to the next leaf can be stored in a compact way.

That gives a compact frontier:

- `len` is the next leaf index,
- `peaks` are the full left subtrees on the path to `len`,
- empty subtrees on the right are implied,
- the public commitment is the Merkle root of that path.

With that model, the change is simple:

- keep the append-only shape,
- keep the `len + peaks` frontier,
- stop using `hash(peaks)` as the public commitment,
- use one normal Merkle root instead,
- verify reads with normal Merkle proofs.

This puts `len` inside the commitment instead of leaving it as side data that every caller must handle with care.

# why

- The current design commits to `hash(peaks)`, not to the full append-only tree shape.
- That makes `len` a special case.
- The rooted form is easier to reason about because the root already fixes the shape.
- Reads become plain Merkle verification.
- Appends still use the same cheap binary-carry update.

# model

For `len = 13 = 1101₂`, the path to the next leaf looks like this:

```mermaid
flowchart BT
E0["empty leaf at index 13"] --> N0
P1["peak(1)"] --> N0
N0 --> N1
E2["empty(2)"] --> N1
N1 --> N2
P4["peak(4)"] --> N2
N2 --> N3
P8["peak(8)"] --> N3
N3 --> R["root"]
```

Each bit of `len` tells you which side is full:

- bit `1`: a full subtree is on the left, so consume one peak,
- bit `0`: the sibling on the right is an empty subtree.

# algorithm

`append(leaf)` is still the same binary carry now used by `Mmr::add()`.

- Start with the new leaf as a subtree of size `1`.
- While the low bit of `len` is `1`, pop the last peak and merge it on the left.
- Push the final merged node as the new last peak.
- Increase `len`.
- Recompute or cache the single Merkle root for the new frontier.

`root()` is computed by folding the path to `len`.

```rust
pub struct MerkleFrontier {
len: usize,
peaks: Vec, // same order as MmrPeaks: largest -> smallest
}

impl MerkleFrontier {
pub fn root(&self) -> Word {
if self.len == 0 {
return empty_subtree_root(0);
}

let mut bits = self.len;
let mut level = 0u8;
let mut peak_idx = self.peaks.len();
let mut acc = empty_subtree_root(0);

while bits != 0 {
if bits & 1 == 1 {
peak_idx -= 1;
acc = merge(self.peaks[peak_idx], acc);
} else {
acc = merge(acc, empty_subtree_root(level));
}

bits >>= 1;
level += 1;
}

acc
}

pub fn append(&mut self, mut node: Word) {
let mut n = self.len;
while n & 1 == 1 {
let left = self.peaks.pop().expect("peak must exist");
node = merge(left, node);
n >>= 1;
}
self.peaks.push(node);
self.len += 1;
}
}
```

If a protocol wants a fixed tree depth, keep folding with empty roots after the last set bit.

# API shape

The public API can look like a normal Merkle API:

| Today | Proposed |
|---|---|
| `MmrPeaks::hash_peaks()` | `MerkleFrontier::root()` |
| `MmrPeaks::verify(value, MmrProof)` | `MerklePath::verify(index, value, &root)` |
| `MmrProof` | plain `MerkleProof` or `(index, value, MerklePath)` |
| `MmrPeaks` as public state | `MerkleFrontier` as public state |
| `Forest` as public version tag | `len` as public version tag |

The frontier can still stay peak-based inside. The real change is at the API edge and in the commitment.

# migration plan

- If a consumer only stores a commitment, switch from `hash(peaks)` to `root`.
- If a consumer verifies reads, switch from `MmrProof` to standard `MerklePath` verification.
- If a consumer appends, store `len + peaks` behind a small wrapper and keep the current carry logic.
- If a consumer produces proofs, keep the current `open()` logic inside first, but change the API edge to return plain Merkle proofs against `root`.
- If a consumer syncs partial state, keep the existing delta logic first and change the public commitment first.
- If a consumer tracks a subset of leaves, base the next type on `PartialMmr`, but make the public anchor a single root, not `hash(peaks)`.

One clean rollout is:

1. Add `MerkleFrontier` next to the current MMR types.
2. Add `root()` and plain Merkle proof verification.
3. Mark `hash_peaks()` as legacy.
4. Move public callers to the rooted API.
5. Remove peak-hash commitments after downstream code is off them.

# facade sketch

A small facade can bridge the two APIs:

```rust
pub struct MerkleFrontierFacade {
inner: MmrPeaks,
}

impl MerkleFrontierFacade {
pub fn num_leaves(&self) -> usize {
self.inner.num_leaves()
}

pub fn root(&self) -> Word {
MerkleFrontier {
len: self.inner.num_leaves(),
peaks: self.inner.peaks().to_vec(),
}
.root()
}

pub fn into_legacy_peaks(self) -> MmrPeaks {
self.inner
}
}
```

That gives downstream code a clear path:

- first change the commitment type,
- then change proof production and verification,
- then drop the peak-hash API.

# code pointers in `0xMiden/crypto`

Pointers below use `0xMiden/crypto@68ac2cf0b8473a63998b38149f8d599ffe3179b8`.

- Current append carry logic in full MMR: [`miden-crypto/src/merkle/mmr/full.rs#L153-L185`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/full.rs#L153-L185)
- Current proof opening and peak extraction: [`miden-crypto/src/merkle/mmr/full.rs#L116-L215`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/full.rs#L116-L215)
- Current peak-hash commitment and peak-local verification: [`miden-crypto/src/merkle/mmr/peaks.rs#L120-L175`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/peaks.rs#L120-L175)
- Current proof types: [`miden-crypto/src/merkle/mmr/proof.rs#L10-L163`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/proof.rs#L10-L163)
- Current path builder inside the full MMR: [`miden-crypto/src/merkle/mmr/full.rs#L315-L378`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/full.rs#L315-L378)
- Current partial-state append and tracking logic: [`miden-crypto/src/merkle/mmr/partial.rs#L304-L445`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/partial.rs#L304-L445)
- Current partial-state delta apply logic: [`miden-crypto/src/merkle/mmr/partial.rs#L514-L643`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/partial.rs#L514-L643)
- Bit-level forest helpers that already encode the frontier shape: [`miden-crypto/src/merkle/mmr/forest.rs#L12-L110`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/forest.rs#L12-L110) and [`miden-crypto/src/merkle/mmr/forest.rs#L308-L445`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/mmr/forest.rs#L308-L445)
- Standard Merkle proof verification to reuse: [`miden-crypto/src/merkle/path.rs#L58-L86`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/path.rs#L58-L86)
- Empty subtree roots to reuse for the frontier path: [`miden-crypto/src/merkle/empty_roots.rs#L8-L33`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/empty_roots.rs#L8-L33)
- Store integration that may want a new rooted adapter later: [`miden-crypto/src/merkle/store/mod.rs#L533-L537`](https://github.com/0xMiden/crypto/blob/68ac2cf0b8473a63998b38149f8d599ffe3179b8/miden-crypto/src/merkle/store/mod.rs#L533-L537)

# scope

This does not ask for a new general tree.

- It keeps the same append-only frontier model.
- It keeps the same cheap updates.
- It only narrows the public model to the rooted frontier described above.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.