erigontech / erigontech/erigon

cl/persistence/blob_storage: flat 10k-slot bucket directories reach 1.28M entries on a supernode

Open
#23,426 0 comments 0 reactions 0 assignees View on GitHub
Caplin
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

Both hot sidecar stores put every file in one flat directory per 10,000 slots: `/_` (`cl/persistence/blob_storage/blob_db.go:44-46,71-77`, `data_column_db.go:62-68`).

That directory holds up to 90,000 blob files, or 1,280,000 column files on a supernode custodying all 128 columns.

Pruning is fine — it only lists bucket names. The per-object paths are not: `GetSavedColumnIndex` and `RemoveAllColumnSidecars` each do `NumberOfColumns` `Stat`/`Remove` calls inside that directory, per availability check and per removal (`data_column_db.go:154-163,197-212`).

Fix: a per-slot leaf, `//_` — at most 9 blob or 128 column files per directory. It also lets the retention floor be exact; today the cutoff rounds down to a bucket boundary and keeps up to 9,999 extra slots.

Migration has to be an in-place rename walk, not a delete-and-refill. These files are never torrented — `db/downloader/` has no reference to either directory — and `erigon snapshots reset` is scoped to `SnapDir` (`db/datadir/reset/reset.go:43,131`), so there is no canonical copy to re-fetch; refill is p2p backfill, bounded by the retention window. Under `--caplin.blobs-archive` without `--caplin.snapgen` the hot store is the only copy of everything below that window and peers will not serve it back.

`Dirs.RenameOldVersions` (`db/datadir/dirs.go:275-347`) already walks both directories at startup and renames in place, so the hook exists; it currently matches only a `vX-` filename prefix. Note #23429 — that walk is pure waste today.

Blocked on #23413 — after the collapse this is one path function plus the pruner, not the same edit twice.

Part of #23024, item 4.

Contributor guide

Open the contributing guide

Research direction

Read cl/persistence/blob_storage/blob_db.go, data_column_db.go, and Dirs.RenameOldVersions in db/datadir/dirs.go, starting with the existing path construction and startup rename walk. Coordinate with blocked issue #23413. Done means the in-place migration preserves files, the new per-slot layout limits directory size, and retention no longer rounds down to a bucket boundary.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, databases, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.