erigontech / erigontech/erigon

Decentralized snapshot distribution via discv5 + BitTorrent

Open
#19,660 2 comments 2 reactions 0 assignees View on GitHub
ErigonDB headliner Networking
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

## Goal

Trustlessly distribute erigon chain and state snapshots without relying on centralized infrastructure (GitHub repos, R2 buckets).

## Problem

Today, snapshot info-hashes are distributed via a centralized path:
1. The `erigon-snapshot` GitHub repo maintains canonical hashes
2. These are embedded into the binary as `preverified.toml`
3. At runtime, nodes fall back to fetching from R2/GitHub if no local copy exists

This creates a single point of failure and a central trust dependency for the entire snapshot distribution pipeline.

## Solution

Replace the centralized info-hash distribution with a peer-to-peer mechanism:

1. **Publish**: Each node creates a BitTorrent torrent of its `preverified.toml` and advertises the torrent info-hash via a custom discv5 ENR entry
2. **Discover**: New nodes find peers via discv5, read their ENR entries to get the latest info-hash, and download `preverified.toml` via BitTorrent
3. **Bootstrap**: Once a node has `preverified.toml`, the existing downloader pipeline takes over to fetch actual snapshots

### Key Design Properties

- **Deterministic**: `preverified.toml` is byte-identical across nodes at the same snapshot step, producing a stable torrent info-hash
- **Versioned**: The max frozen transaction number provides monotonic ordering — a new deterministic file is produced for every max step
- **Append-only**: Each new version is a strict superset of the previous, enabling cheap integrity checks
- **Backward compatible**: Nodes that don't support the new ENR entry fall back to the existing centralized path
- **Incrementally trustable**: Trust layers are additive — POC works without trust, statistical consensus adds basic protection, UCAN adds cryptographic guarantees

## Implementation Steps

### Step 1: POC — Physical Distribution Flow
- #19657

Prove the mechanics: ENR publishing, torrent creation, peer discovery, download, and integration with the existing downloader pipeline. No trust layer — works on honest-network assumption.

### Step 2: Statistical Trust — Threshold Consensus
- #19658

Add lightweight production trust: collect info-hashes from multiple peers, require majority agreement before accepting. Defeats isolated bad actors. Does not require any cryptographic infrastructure.

### Step 3: Identity Trust — UCAN Delegation
- #19659

Add cryptographic identity trust using UCAN delegation chains. Known trusted publishers sign their updates, nodes verify locally. Eliminates the honest-majority assumption without introducing a central runtime dependency. Defeats Sybil attacks.

## Architecture

```
┌─────────────────────────────────┐
│ Node Startup │
└──────────────┬──────────────────┘

┌──────────────▼──────────────────┐
│ Create torrent of │
│ preverified.toml │
│ (deterministic info-hash) │
└──────────────┬──────────────────┘

┌──────────────▼──────────────────┐
│ Set ENR entry: │
│ {FrozenTx, InfoHash} │
│ (28 bytes, fits in 300B limit) │
└──────────────┬──────────────────┘

┌────────────────────┼────────────────────┐
│ │ │
┌─────────▼─────────┐ ┌──────▼──────┐ ┌──────────▼─────────┐
│ Seed torrent │ │ discv5 │ │ Existing │
│ via BitTorrent │ │ discovery │ │ snapshot │
│ │ │ ← peers │ │ downloader │
└────────────────────┘ └──────┬──────┘ └──────────▲─────────┘
│ │
┌──────▼──────┐ │
│ Read peer │ │
│ ENR entries│ │
└──────┬──────┘ │
│ │
┌──────▼──────┐ │
│ Download │ │
│ preverified│──────────────┘
│ .toml via │
│ BitTorrent │
└─────────────┘
```

## Trust Layers (additive)

| Layer | Mechanism | Threat Model | Dependency |
|-------|-----------|-------------|------------|
| None (POC) | First peer wins | Honest network | None |
| Statistical | Threshold consensus (N-of-M) | Honest majority | None |
| UCAN | Cryptographic delegation chains | Byzantine (Sybil-resistant) | Root key in binary |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.