erigontech / erigontech/erigon
downloader: persist preverified.toml at sync start and write sync-completion marker at end
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
## Problem
The preverified set currently has no durable on-disk record tied to the lifecycle of a sync. This creates two gaps:
1. **At sync start**: if the preverified set was obtained from a remote or embedded source, it is not written to disk, so the next run may fetch a different (newer) set and silently diverge from what was actually downloaded.
2. **At sync end**: there is no marker indicating that sync completed. Without this, the node cannot distinguish "sync in progress / interrupted" from "sync completed but directory subsequently modified by normal operation".
## Context
There used to be a dedicated file that indicated sync completion; this was removed at some point. Currently, `preverified.toml` is written at the **end** of a sync and serves as an implicit completion marker — its presence signals that sync finished. This issue proposes inverting that: write `preverified.toml` at the **start** of sync (pre-commit it) so it anchors the chosen preverified set from the beginning, and introduce a separate explicit completion marker.
## Proposed behaviour
### At the start of sync
When the preverified set is resolved (from local, remote, or embedded source):
- Write it to `preverified.toml` in the snapshot directory if it is not already present, or if the resolved set exactly matches what is already on disk (no-op in the match case — the user is getting what they expect).
- If the resolved set **differs** from an existing `preverified.toml`, treat this as a source-selection conflict and surface it clearly (see #19714).
This ensures the default source for subsequent runs becomes `local`, giving stable, reproducible behaviour across restarts.
### At the end of sync
Write a marker file (e.g. `sync-complete`) to the snapshot directory when the initial sync finishes successfully. This file signals that:
- The snapshot directory reached a known-good state at least once.
- Subsequent modifications (merges, retirements, step-size changes) are expected and do not invalidate the directory.
The extraneous-file check in #19716 should only apply when this marker is **absent** (i.e. sync never completed). Once the marker exists, files may change in ways that are valid and unpredictable without being wrong.
## Related issues
- #19716 — error on extraneous snapshots when resuming incomplete sync (uses the completion marker)
- #19715 — record and validate download filters
- #19714 — preverified source selection on main node
- #19669 — issue 1: stale `preverified.toml` prevents re-download after file deletion
- #19435 — canonical clean state via snapshot reset; complement to persisting preverified.toml
Contributor guide
Assessment
This issue has not been assessed yet.