Partial Statefulness for Erigon: EIP-7928 + Sparse Snapshots

# Partial Statefulness for Erigon: EIP-7928 + Sparse Snapshots

## Context

[Geth PR #33764](https://github.com/ethereum/go-ethereum/pull/33764) introduces [EIP-7928](https://eips.ethereum.org/EIPS/eip-7928) "partial statefulness" — nodes keep complete accounts but only sync storage/code for configured contracts, using Block Access Lists (BAL) to apply state diffs without re-executing transactions. Storage drops from ~640 GiB to ~59 GiB (≈ 91% reduction).

Erigon already has two of the three pieces needed to match this:

1. **BAL support** — `execution/stagedsync/bal_create.go` already processes Block Access Lists decoded from `NewPayloadV5` in the Engine API.
2. **Sparse snapshots POC** ([`poc/sparse-snapshots`](https://github.com/erigontech/erigon/tree/poc/sparse-snapshots)) — domain `.kv` files served on-demand over BitTorrent, indices/accessors stored locally.

This issue tracks the work to combine these into Erigon's equivalent of geth's partial-state node, using sparse snapshots as the state backend instead of geth's filtered trie approach.

### Related work

- [#20526 — decentralized snapshot distribution via chain.toml P2P discovery](https://github.com/erigontech/erigon/pull/20526): extends Erigon's BitTorrent snapshot distribution with decentralized peer discovery. Partial-state nodes inherit this serving substrate directly — the sparse `.kv` torrents a partial node registers as on-demand metadata sources benefit from the same discovery path. Shipping these together means partial nodes can locate seeders for untracked-contract data without relying on static trackers or centralised WebSeed CDNs.

## Key Architectural Difference vs Geth

| Aspect | Geth (PR #33764) | Erigon (this plan) |
|---|---|---|
| State backend | Modified Patricia Merkle Trie + filtered snap sync | Sparse domain files via BitTorrent on-demand reads |
| What's local | Full account trie + tracked-contract storage | Local indices + tracked-contract domains in MDBX |
| What's remote | Nothing (just not synced) | Untracked domain `.kv` data served from torrent peers |
| State root | Tolerates divergence, chains from computed root | Same as geth initially — trie relaxation for untracked contracts. QMTree is an optional follow-on to tighten the correctness anchor. |
| Sync approach | Modified snap sync with skip markers | Blacklist domain `.kv` from download, register as sparse torrents |
| BAL role | Primary state update mechanism | State update + torrent piece prefetch oracle |
| Transport | devp2p snap/1→snap/2 | Standard BitTorrent (already deployed) |

Erigon does not implement the devp2p snap protocol at all — the P2P layer only supports eth/68–70. All state distribution already goes over BitTorrent + WebSeed CDN. The sparse snapshots POC extends this to selective on-demand piece reads, which is the serving mechanism we'll reuse here.

## Geth → Erigon File Mapping

### Configuration & CLI

| Geth File | What It Does | Erigon Equivalent | Status |
|---|---|---|---|
| `cmd/geth/main.go` | Registers partial state flags | `cmd/erigon/main.go` / flag config | To build |
| `cmd/utils/flags.go` | `--partial-state[.contracts|.contracts-file|.bal-retention|.chain-retention]` | Erigon flag system | To build |
| `eth/ethconfig/config.go` | `PartialStateConfig`, `LoadPartialStateContracts()` | `ethconfig.Config` extension | To build |

### Core State Management

| Geth File | What It Does | Erigon Equivalent | Status |
|---|---|---|---|
| `core/state/partial/filter.go` | `ContractFilter` interface | `db/state/contract_filter.go` (new) | To build |
| `core/state/partial/filter_test.go` | Filter tests | Corresponding test file | To build |
| `core/state/partial/history.go` | BAL history manager (wraps rawdb) | Extend `rawdb.ReadBlockAccessListBytes` with retention | Partially exists |
| `core/state/partial/state.go` | `ApplyBALAndComputeRoot()`, `ProcessBlockWithBAL()`, `HandlePartialReorg()` — three-phase commit | `bal_create.go` `ApplyBALDiffs()` (new) + existing `ProcessBAL` | To build on existing |
| `core/state/partial/state_test.go` | State tests (1126 lines) | `db/state/sparse_state_test.go` pattern from POC | To build |

### Blockchain Integration

| Geth File | What It Does | Erigon Equivalent | Status |
|---|---|---|---|
| `core/blockchain.go` | Partial state processing, chain retention, reorg | `execution/stagedsync/exec3_parallel.go` + stage unwind | Modify existing |
| `core/blockchain_partial.go` | BAL-based block processing, gap blocks, canonical-hash backfill | Extend `execution/stagedsync/bal_create.go` | To build on existing |
| `core/blockchain_partial_test.go` | Integration tests | New test file | To build |

### Database & Storage

| Geth File | What It Does | Erigon Equivalent | Status |
|---|---|---|---|
| `core/rawdb/accessors_bal.go` | Read/write/delete/prune BAL (RLP, `"p" + blockNum`) | `rawdb.ReadBlockAccessListBytes` / `WriteBlockAccessListBytes` | Exists |
| `core/rawdb/chain_freezer.go` | `SetChainRetention()` rolling window | Snapshot pruning + `--prune.mode=sparse` | Partially exists |
| `core/rawdb/schema.go` | BAL history key prefix | `db/kv/tables.go` | Partially exists |

### Snap Synchronization

| Geth File | What It Does | Erigon Equivalent | Status |
|---|---|---|---|
| `eth/protocols/snap/sync_partial.go` | Skip markers, `shouldSyncStorage/Code` | **Not needed** — BitTorrent sparse snapshots | N/A |
| `eth/downloader/downloader.go` | Filter in sync pipeline | `db/snapshotsync/snapshotsync.go` blacklist + `db/state/domain.go` sparse lookup | Exists in POC |
| `eth/handler.go` | Wires filter into handler | `node/eth/backend.go` sparse wiring | Exists in POC |

### RPC & API

| Geth File | What It Does | Erigon Equivalent | Status |
|---|---|---|---|
| `internal/web3ext/web3ext.go` | RPC method definitions | Erigon RPC method definitions | To modify |
| Various RPC files | `GetStorageAt` → `-32001`, `GetCode` → `-32002`, `GetProof` restricted, `Call`/`EstimateGas` error for untracked | `rpc/jsonrpc/` handlers | To modify |

### Engine API

| Geth File | What It Does | Erigon Equivalent | Status |
|---|---|---|---|
| `eth/catalyst/api.go` | `NewPayloadV5` accepts BAL data | `execution/engineapi/engine_server.go` lines 302–337 | **Exists** |

## Concept Mapping

| Geth Concept | Erigon Equivalent | Notes |
|---|---|---|
| Complete account trie | `AccountsDomain` (full) | Both keep all accounts |
| Selective storage sync | `StorageDomain` (filtered writes) | Geth filters at trie level; Erigon at domain write level |
| Selective code sync | `CodeDomain` (filtered writes) | Same pattern |
| BAL from `NewPayloadV5` | Already implemented | `engine_server.go` decodes, `bal_create.go` processes |
| BAL history for reorg | `rawdb.ReadBlockAccessListBytes` | Exists; add retention-window pruning |
| Modified snap sync | BitTorrent sparse snapshots | Fundamentally different — no snap protocol needed |
| Skip markers | Sparse torrent registration | Geth marks skipped storage in DB; Erigon registers `.kv` as sparse torrents |
| State root recomputation | Trie relaxation (same as geth) or `--sparse.no-commitment` | Initial impl matches geth's divergence tolerance; QMTree is a later option |
| `ContractFilter` interface | `ContractFilter` interface (new) | Direct port, same semantics |
| `ConfiguredFilter` | `StaticContractFilter` (new) | Same concept, Erigon address format |
| `PartialState` manager | `FilteredStateWriter` + `ApplyBALDiffs()` | Split across writer and BAL application |
| Peer storage root queries | Not needed | Sparse torrent read instead |
| `ErrDeepReorg` | Stage unwind + resync trigger | Existing unwind handles shallow; deep triggers resync |
| Chain retention window | `--prune.mode=sparse` + chain retention config | Partially exists in POC |

### Doesn't Need Porting

| Geth Component | Why Not |
|---|---|
| `sync_partial.go` skip markers | BitTorrent sparse snapshots, not snap sync |
| Storage root resolution from peers | Sparse torrent reads on-demand |
| Three-phase trie commit ordering | Erigon's domain storage has no trie ordering constraints |
| `PartialStateSync()` healing | No trie healing — indices local, data on-demand |
| `AdvancePartialHead()` | Stage pipeline handles head advancement |
| Stale pivot recovery | Snapshot sync doesn't use pivot model |
| Canonical hash backfilling | Snapshot infrastructure handles block indexing |

### Erigon-Specific Work (No Geth Equivalent)

| Need | Why |
|---|---|
| BAL-driven torrent prefetch | Parse BAL → map to torrent pieces → parallel pre-request |
| QMTree leaf generation from BAL (optional, follow-on) | `AppendFromBAL()` for untracked-state correctness verification — not required for the initial implementation |
| Sparse domain file blacklist filtering | Contract-filter-aware blacklist (POC is all-or-nothing) |
| `FilteredStateWriter` | Domain write filtering by contract (geth filters at trie level) |

## Proposed Action Plan

### Phase 1 — Contract Filter & Configuration

New file `db/state/contract_filter.go`:

```go
type ContractFilter interface {
IsTracked(addr common.Address) bool
ShouldSyncStorage(addr common.Address) bool
ShouldSyncCode(addr common.Address) bool
}
```

CLI flags:
- `--partial-state` (bool)
- `--partial-state.contracts` (comma-separated addresses)
- `--partial-state.contracts-file` (JSON path)
- `--partial-state.bal-retention` (uint64, default 256)
- `--partial-state.chain-retention` (uint64, default 1024)

Wire filter into `ExecuteBlockCfg`, snapshot config, RPC server.

### Phase 2 — Sparse State with Contract Filtering

Extend the sparse POC to be contract-filter-aware so tracked contracts get full local domains while untracked contracts use sparse torrent reads.

- **`FilteredStateWriter`** wraps `StateWriter`: tracked → local MDBX; untracked → skip local writes; `AccountsDomain` always proceeds.
- **Selective blacklist** in `db/snapshotsync/snapshotsync.go`. Note: Erigon's domain files are not account-sharded — a single `v1-storage.0-64.kv` contains storage for all contracts in that step range. The filter therefore applies at the write path (don't persist untracked state) and read path (fall through to sparse torrent), not at file-download granularity.
- **Domain read path** (`db/state/domain.go` `getLatestFromFile`): tracked → local BTree; untracked → sparse torrent read.

### Phase 3 — BAL-Driven Execution (No Re-Execution for Untracked)

- **`ApplyBALDiffs()`** (new in `bal_create.go`): input `BAL + ContractFilter + SharedDomains`; apply balance/nonce to `AccountsDomain` for all; apply storage/code to `Storage`/`CodeDomain` only for tracked contracts.
- **Modified execution stage** (`exec3_parallel.go`): examine BAL per block; tracked-only txs → execute normally; mixed/untracked txs → apply from BAL via `ApplyBALDiffs()`.
- **BAL-driven torrent prefetch** (`sparse_adapter.go`): parse BAL → BTree offsets → torrent piece indices → parallel piece requests. Target: 5m30s → 5–10s per block cold.

### Phase 4 — Commitment & Verification

**Primary approach — trie relaxation (same as geth PR #33764)**: compute the PMT root for tracked contracts only, accept root divergence for untracked contracts, chain from the locally-computed root via an atomic pointer. This is what geth ships in PR #33764 and is sufficient to get partial-state nodes running without blocking on other work.

Also available: `--sparse.no-commitment` (from the sparse POC) to skip commitment entirely for use cases that carry their own proofs (L2 / bridge nodes).

**Follow-on: QMTree as a tighter correctness anchor (optional, separate issue).** Once the [QMTree POC](https://github.com/erigontech/cocoon/tree/master/pocs/qmtree) is production-ready, it can replace the divergence-tolerant PMT root with an anchor that commits to *all* state changes per tx — including untracked contracts — via BAL-driven leaves. Leaf format: `preStateHash || stateChangeHash || transitionHash || previousLeafHash`; `transitionHash` = sentinel for BAL-applied txs. This is strictly additive to Phase 4 and is **not** on the critical path for an initial partial-state node.

### Phase 5 — RPC Awareness

| Method | Tracked | Untracked |
|---|---|---|
| `eth_getBalance` / `eth_getTransactionCount` | Normal | Normal (accounts always tracked) |
| `eth_getStorageAt` | Normal | Error `-32001` |
| `eth_getCode` | Normal | Error `-32002` |
| `eth_getProof` (account) | Normal | Normal |
| `eth_getProof` (storage) | Normal | Error `-32001` |
| `eth_call` / `eth_estimateGas` | Normal | Error if touches untracked |

Plus: `eth_getPartialStateInfo` (new) returns tracked contracts and config. If QMTree is integrated later, `qm_*` RPC methods would provide proof-based access for untracked-state verification — not part of the initial implementation.

### Phase 6 — Reorg Handling

- **Shallow reorgs** (within BAL retention, default 256): unwind all domains to fork point (existing mechanism), re-apply BAL diffs from stored BAL history. If QMTree is integrated later, the unwind also calls `Tracker.Unwind(targetTxNum)`.
- **Deep reorgs** (beyond retention): return error (geth's `ErrDeepReorg`), trigger partial re-sync (download missing BAL from peers / eth/71 when available), fallback to full snapshot re-sync.
- **Chain retention enforcement**: rolling window for bodies/receipts (default 1024 blocks); older blocks keep headers only. Maps to geth's `ChainFreezer.SetChainRetention()`.

### Phase 7 — Prune Mode Integration

- `--prune.mode=sparse` + `--partial-state` combined: sparse prune handles domain file blacklisting; partial state filter handles contract-specific domain writes; BAL retention controls reorg recovery depth.
- Caplin sparse support already added in POC for consensus layer snapshots.

## Deferred: Snap Protocol Server

Adding devp2p snap protocol server support so Erigon can respond to `GetAccountRange`, `GetStorageRanges`, etc. is deferred.

- **Full snap server** (~6–8 weeks): `GetAccountRange`/`GetStorageRanges`/`GetByteCodes` ~1–2 weeks (data available via `RangeAsOf()` + `Witness()`); `GetTrieNodes` ~3–4 weeks (Erigon doesn't store intermediate MPT nodes — requires on-the-fly reconstruction from commitment domain).
- **Partial snap server** (~1–2 weeks): implement only `GetAccountRange`/`GetStorageRanges`/`GetByteCodes`, skip `GetTrieNodes`. Viable once snap/2 standardises and clients stop requiring `GetTrieNodes` for trie healing (geth's PR #33764 is targeting exactly this). Would give cross-client partial-state sync (geth EIP-7928 nodes syncing from Erigon) without the full-server complexity.

BitTorrent-based sparse snapshots already solve Erigon-to-Erigon partial state serving without any protocol work. The snap server adds cross-client interop and isn't needed for the initial implementation.

## Critical Files Reference

| File | Role | Branch |
|---|---|---|
| `execution/stagedsync/bal_create.go` | BAL creation/processing — extend with `ApplyBALDiffs` | main |
| `execution/stagedsync/exec3_parallel.go` | Execution loop — add partial state mode | main |
| `execution/engineapi/engine_server.go` | Engine API — BAL already decoded here | main |
| `db/state/domain.go` | Domain read/write — `SparseTorrentLookup` interface | poc/sparse-snapshots |
| `db/downloader/sparse_adapter.go` | Torrent reader creation, metadata registration | poc/sparse-snapshots |
| `db/snapshotsync/snapshotsync.go` | Blacklist building for sparse files | poc/sparse-snapshots |
| `db/state/aggregator.go` | `OpenSparseFiles()` injects FilesItem entries | poc/sparse-snapshots |
| `db/state/dirty_files.go` | Sparse flag, visibility bypass | poc/sparse-snapshots |
| `db/kv/prune/storage_mode.go` | `--prune.mode=sparse` support | poc/sparse-snapshots |
| `node/eth/backend.go` | Wiring: metadata registration, sparse file opening | poc/sparse-snapshots |

## Dependencies

- **[Sparse snapshots POC](https://github.com/erigontech/erigon/tree/poc/sparse-snapshots)** — provides `SparseTorrentLookup`, sparse domain file registration, on-demand torrent reads, `--prune.mode=sparse`. Data serving layer.
- **BAL support (main)** — `bal_create.go` `ProcessBAL`, Engine API `NewPayloadV5` BAL decoding. State update mechanism.
- **[#20526 — decentralized snapshots](https://github.com/erigontech/erigon/pull/20526)** — related work. Not a hard dependency, but ships the P2P peer-discovery substrate that sparse `.kv` torrents plug into. Pairing the two lets partial-state nodes find seeders for untracked-contract data without centralised trackers.

### Optional / Follow-on

- **[QMTree](https://github.com/erigontech/cocoon/tree/master/pocs/qmtree)** — not required for the initial implementation. The first version matches geth PR #33764's approach (trie relaxation + atomic-pointer chaining). QMTree is a follow-on to tighten the correctness anchor for untracked contracts, tracked as a separate issue once the QMTree POC lands.

## Open Questions

1. **Merge behaviour**: the sparse POC disables merging (`--snap.state.stop`). For long-running partial state nodes, how do we handle aggregator merges for tracked-contract domains without breaking sparse file consistency?
2. **Mixed execution**: when a tx touches both tracked and untracked contracts, do we execute it fully (requires sparse reads for untracked state) or apply from BAL? BAL is simpler; full execution preserves EVM traces (useful if/when QMTree is integrated).

---

Source docs (internal):
- [`partial-state/docs/design.md`](https://github.com/erigontech/cocoon/blob/master/pocs/partial-state/docs/design.md)
- [`partial-state/docs/geth-mapping.md`](https://github.com/erigontech/cocoon/blob/master/pocs/partial-state/docs/geth-mapping.md)