erigontech / erigontech/erigon
snapshots: verify local blob sidecars and repair from official snapshots
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
# Rationale
Operators currently have several low-level recovery paths: downloader piece verification, snapshot reset, index rebuild, and broad snapshot removal/re-sync. There is no focused user workflow that answers whether local blob-sidecar snapshots are complete and semantically valid, repairs official files from the official snapshot distribution, and clearly fails when the requested history is unavailable.
Torrent hashes detect byte-level divergence from an official file, but they do not by themselves establish slot coverage, expected sidecar counts, canonical-root correctness, or KZG validity. Operators should not need to infer blob completeness from filenames or directory presence.
This issue covers restoration of an operator datadir from official snapshots. Snapshot producer repair from peers and publication hard-faulting are tracked separately.
# Implementation
Add a user-facing verify-and-repair workflow to the main `erigon snapshots` command family:
- Inspect the configured datadir under its exclusive lock and determine the expected official blob snapshot set from the chain's preverified manifest.
- Validate file/range presence, torrent piece hashes, segment readability, index-to-segment consistency, canonical slot/root mapping, expected sidecar counts/indexes, and KZG proofs.
- Produce actionable diagnostics that identify which files or ranges are missing, corrupt, locally generated, or not available in the official snapshot set.
- In repair mode, remove or quarantine only the affected official artifacts and ask the existing downloader to reacquire them from the official snapshot distribution.
- Rebuild derived indexes locally when the segment data is valid and only the index is missing or corrupt.
- Re-run verification after download/rebuild and exit non-zero if completeness cannot be established.
- Preserve locally generated snapshots by default unless the user explicitly opts into replacing them.
- Provide a dry-run mode that performs no mutation.
This should reuse the existing preverified/downloader/reset machinery rather than introduce a second distribution protocol. It may share the semantic blob verifier with the snapshotter workflow, but its repair source and safety policy are different.
## Acceptance criteria
- The command is built as part of the main `erigon` binary and operates through `erigon snapshots`.
- Verify-only mode never modifies the datadir and exits non-zero for missing or invalid required data.
- Repair mode downloads only affected official files/pieces where possible, rebuilds derived indexes, and passes a second verification.
- Missing official coverage is reported explicitly; it is never treated as a successful repair.
- Local-only files are preserved by default.
- Interrupted repair is restart-safe and cannot expose a partially replaced segment set.
- Tests cover missing files, truncated/corrupt pieces, valid segment with missing index, semantic sidecar gaps despite valid filenames, unsupported ranges, and interrupted repair.
## Related
- #23024 tracks Caplin snapshot lifecycle and integrity parity.
- #23224 tracks a missing-sidecar antiquation wedge.
- #23909 shows that operators currently lack a reliable way to establish and restore historical blob completeness.
- #23911 tracks the snapshotter peer-repair and publication hard-fault workflow.
Contributor guide
Assessment
This issue has not been assessed yet.