erigontech / erigontech/erigon
Decentralized snapshot distribution via discv5 + BitTorrent
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
## Goal
Trustlessly distribute erigon chain and state snapshots without relying on centralized infrastructure (GitHub repos, R2 buckets).
## Problem
Today, snapshot info-hashes are distributed via a centralized path:
1. The `erigon-snapshot` GitHub repo maintains canonical hashes
2. These are embedded into the binary as `preverified.toml`
3. At runtime, nodes fall back to fetching from R2/GitHub if no local copy exists
This creates a single point of failure and a central trust dependency for the entire snapshot distribution pipeline.
## Solution
Replace the centralized info-hash distribution with a peer-to-peer mechanism:
1. **Publish**: Each node creates a BitTorrent torrent of its `preverified.toml` and advertises the torrent info-hash via a custom discv5 ENR entry
2. **Discover**: New nodes find peers via discv5, read their ENR entries to get the latest info-hash, and download `preverified.toml` via BitTorrent
3. **Bootstrap**: Once a node has `preverified.toml`, the existing downloader pipeline takes over to fetch actual snapshots
### Key Design Properties
- **Deterministic**: `preverified.toml` is byte-identical across nodes at the same snapshot step, producing a stable torrent info-hash
- **Versioned**: The max frozen transaction number provides monotonic ordering — a new deterministic file is produced for every max step
- **Append-only**: Each new version is a strict superset of the previous, enabling cheap integrity checks
- **Backward compatible**: Nodes that don't support the new ENR entry fall back to the existing centralized path
- **Incrementally trustable**: Trust layers are additive — POC works without trust, statistical consensus adds basic protection, UCAN adds cryptographic guarantees
## Implementation Steps
### Step 1: POC — Physical Distribution Flow
- #19657
Prove the mechanics: ENR publishing, torrent creation, peer discovery, download, and integration with the existing downloader pipeline. No trust layer — works on honest-network assumption.
### Step 2: Statistical Trust — Threshold Consensus
- #19658
Add lightweight production trust: collect info-hashes from multiple peers, require majority agreement before accepting. Defeats isolated bad actors. Does not require any cryptographic infrastructure.
### Step 3: Identity Trust — UCAN Delegation
- #19659
Add cryptographic identity trust using UCAN delegation chains. Known trusted publishers sign their updates, nodes verify locally. Eliminates the honest-majority assumption without introducing a central runtime dependency. Defeats Sybil attacks.
## Architecture
```
┌─────────────────────────────────┐
│ Node Startup │
└──────────────┬──────────────────┘
│
┌──────────────▼──────────────────┐
│ Create torrent of │
│ preverified.toml │
│ (deterministic info-hash) │
└──────────────┬──────────────────┘
│
┌──────────────▼──────────────────┐
│ Set ENR entry: │
│ {FrozenTx, InfoHash} │
│ (28 bytes, fits in 300B limit) │
└──────────────┬──────────────────┘
│
┌────────────────────┼────────────────────┐
│ │ │
┌─────────▼─────────┐ ┌──────▼──────┐ ┌──────────▼─────────┐
│ Seed torrent │ │ discv5 │ │ Existing │
│ via BitTorrent │ │ discovery │ │ snapshot │
│ │ │ ← peers │ │ downloader │
└────────────────────┘ └──────┬──────┘ └──────────▲─────────┘
│ │
┌──────▼──────┐ │
│ Read peer │ │
│ ENR entries│ │
└──────┬──────┘ │
│ │
┌──────▼──────┐ │
│ Download │ │
│ preverified│──────────────┘
│ .toml via │
│ BitTorrent │
└─────────────┘
```
## Trust Layers (additive)
| Layer | Mechanism | Threat Model | Dependency |
|-------|-----------|-------------|------------|
| None (POC) | First peer wins | Honest network | None |
| Statistical | Threshold consensus (N-of-M) | Honest majority | None |
| UCAN | Cryptographic delegation chains | Byzantine (Sybil-resistant) | Root key in binary |
Contributor guide
Assessment
This issue has not been assessed yet.