erigontech / erigontech/erigon
db/downloader: batch-owned download stats instead of one global sample
- Dominant language
- Go
- Stars
- 3.6k
- Forks
- 1.5k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 455
Description
`Downloader.lastStats` is a single slot holding the latest `AggStats`. It is written by the per-batch logging loop (`logDownload` → `storeStats`) and read by `Completed()`, and the samples carry no batch identity.
## Why it matters
`storeStats` holds `BytesCompleted` at the previous sample's level to keep reported progress monotonic, and that max crosses batches, while `BytesTotal` belongs to whichever batch wrote last. Two overlapping batches therefore produce `done > total`, and the snapshot-download progress reporter freezes on a ratio above 1.
#22716 mitigated this by serializing batches: `DownloadSnapshots` holds `downloadBatchLock` for the whole call. That enforces the "one download session at a time" contract the function already documented, but it is coarse — `Download` is a public RPC, and a retry after a client-side transport error can produce a second call, which now queues for the entire download instead of running.
## Proposal
Give each download batch its own counters, have `Completed()` report the active batch's numbers (or report nothing while more than one batch is active), then remove `downloadBatchLock` so concurrent batches are possible again without blending samples.
## Acceptance
Two overlapping `Download` calls never yield `done > total` from `Completed()`, and neither blocks the other.
Follow-up to #22716.
Contributor guide
Research direction
Trace Downloader.lastStats through logDownload, storeStats, and Completed(), then inspect DownloadSnapshots and downloadBatchLock. Define per-batch counters so overlapping Download calls can proceed without mixing samples; done means concurrent calls never make Completed() report done greater than total and neither call blocks the other.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend-api-design, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100