chain.toml V2: versioned snapshot manifest for decentralised archive distribution

# chain.toml V2

## Problem

V1 `chain.toml` is a flat `filename → torrent-hash` map tied to the centralised `preverified.toml` registry. It has no versioning, no coverage metadata, no trust annotations, and no merge awareness. Peers can't exchange manifests intelligently: a node that has merged state into a single large file can't describe its coverage to a peer that hasn't merged, and vice versa, because V1 compares by filename.

The [decentralised snapshot distribution POC](https://github.com/erigontech/erigon/pull/20526) proved peers can find each other via discv5 ENR and seed data directly, but it leaves the format problem open.

There are two reasons to invest in this now — one short-term and tactical, the other long-term and strategic.

In the short term, we have concrete requests for more flexible snapshot distribution from ethpandaops, and for distribution of unsupported chains (e.g. from the Polygon community). Today these either work around V1's centralised preverified registry or fail entirely.

In the long term, assuming the current Ethereum roadmap, the archive will grow significantly. Projections show ~12 PB at 1000× scale under PMT, ~750 TB under qmtree — see [distribution-shape.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/distribution-shape.md). At that scale, peers need to coordinate on coverage, not filenames.

The design covers both cases so we don't close doors that are hard to reopen. Experience with distributed formats shows old versions hang around a very long time — getting the shape right early matters more than shipping every feature at once.

## Timescale

Implementation is currently fill-in work between higher-priority items, so a usable first version will likely arrive incrementally somewhere between 3.6 and 3.9 depending on how priorities shift.

The file format itself is worth reviewing and agreeing **independently** of that timeline so it can be settled before the first version ships.

## Design

V2 is a structured, versioned manifest with explicit sections for block files, metadata, and per-domain state coverage. It carries per-file trust and provenance so consumers can decide what to download and from whom.

**Shape (abridged):**

```toml
version = 2

[blocks]
"v1.0-000000-000500-headers.seg" = "c4e4..."

[meta]
"erigondb.toml" = "49fd..."

[domains.accounts]
coverage = [0, 4096]

[[domains.accounts.files]]
name = "v1.0-accounts.0-2048.kv"
range = [0, 2048]
hash = "abc1..."
trust = "verified"
```

**Key properties:**
- **Versioned** — first key is `version = 2`. Future extensions add fields in place; no V2.1/V2.2 reversioning during dev.
- **Per-domain coverage** — each domain (accounts, storage, code, commitment) advertises the step range its files collectively cover, independent of file boundaries.
- **Canonical layout convergence** — only files at power-of-2 aligned step boundaries appear in the domains section. Two nodes at different merge states publish a comparable subset.
- **Trust model** — `none` (BitTorrent integrity only) | `consensus` (M-of-N peers agree) | `verified` (cryptographic provenance, eventually via segment proofs)
- **ENR extensions** — `DomainSteps` + `MergeDepth` advertise coverage to peers before manifest exchange. V1 nodes decode 3-field ENR entries cleanly (trailing RLP fields default to zero).

## Rollout strategy

The format evolves incrementally. Production stays on V1 until V2 is validated end-to-end.

**Feature flag:** `--snap.manifest.mode=off|dual|v2`
- `off` (default): V1 only — current production behaviour
- `dual`: V1 authoritative, V2 also generated for peers that understand it
- `v2`: V2 preferred when available; V1 fallback

**Version number stays at `2` throughout development.** Each iteration extends the V2 schema in place rather than bumping a sub-version. The feature flag gates the parallel operation; reviewable spec snapshots track the schema evolution.

**Existing flag `--snap.p2p-manifest`** (drives V1 P2P discovery via `StartChainTomlDiscovery`) is superseded by `--snap.manifest.mode`. Behaviour preserved: `mode=dual` and `mode=v2` both enable P2P discovery.

## Scope

Broken into two tracks for review:

- **Sub-issue: baseline (what ships first)** — #20618 — documents V2 as it exists on `main` today (landed via PR #20527): struct, trust, inventory, ENR extensions, canonicity filter. Explicit gap list for what's deferred.
- **Sub-issue: target format** — #20619 — documents the end-state V2: source field, proof roots, publish lifecycle, consumer gap-fill, atomic rotation on merge, prune-window filtering, event-driven coordination.

Both tracks reference the canonical design doc: [chain-toml-v2-format.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/chain-toml-v2-format.md).

## Implementation context

Landed foundations:
- PR #20527 — inventory + trust model + V2 types + ENR extensions (squashed as `d685a28eb8`)
- PR #20526 — decentralised snapshot distribution POC (chain.toml P2P discovery)

In-flight dependencies:
- PR #20471 — downloader component extraction
- PR #20476 — storage component extraction

## Design documents

Published on the `erigontech/erigon-documents` repo under [ethereum/design/erigon-archive/](https://github.com/erigontech/erigon-documents/tree/main/ethereum/design/erigon-archive):

| Doc | Purpose |
|-----|---------|
| [README.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/README.md) | Landing page — start here |
| [requirements.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/requirements.md) | The eight doors — architectural invariants |
| [chain-toml-v2-format.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/chain-toml-v2-format.md) | V2 schema |
| [distribution-shape.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/distribution-shape.md) | Scale projections |
| [distribution-implementation.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/distribution-implementation.md) | Phased delivery plan |
| [storage-component-design.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/storage-component-design.md) | Storage component architecture |
| [txnum-coordination.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/txnum-coordination.md) | Canonical anchor / consistency fence |
| [segment-proof-requirements.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/segment-proof-requirements.md) | Content commitment (for `verified` trust tier) |
| [fork-spec.md](https://github.com/erigontech/erigon-documents/blob/main/ethereum/design/erigon-archive/fork-spec.md) | Partial archive / view semantics |

## Review requests

- Is the feature-flag rollout model (`off|dual|v2`) the right shape?
- Do the trust tiers (`none|consensus|verified`) cover the operational needs?
- Is canonical layout convergence (power-of-2 alignment) the right merge-divergence resolution?
- Any invariants from the eight-doors list missing from the baseline?

## Related work

- **#20587 — Partial Statefulness for Erigon: EIP-7928 + Sparse Snapshots** — this V2 manifest work is a prerequisite. Partial-state nodes distribute sparse `.kv` files over BitTorrent and need per-domain coverage semantics to announce "I serve these contracts' storage for this step range" to peers. The flat V1 format can't express this; V2's `[domains.*]` sections and range-coverage model are the foundation that sparse-snapshot distribution builds on.