ethereum / ethereum/state-actor
besu/nethermind: bound RocksDB index+filter blocks via cache_index_and_filter_blocks (parity with ethrex #117)
- Dominant language
- Go
- Stars
- 12
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
Follow-up from the #117 review.
The besu and nethermind writers leave RocksDB's `cache_index_and_filter_blocks` at its default (`false`), and set **no block cache bound at all** — so every flushed SST's index + bloom-filter blocks are pre-loaded into per-SST table readers, outside any budget, unevictable while the file stays open. With L0 compaction triggers pinned to `MaxInt32` for the whole import (`client/besu/dbs_cgo.go:129-131`, `client/nethermind/dbs_cgo.go:33-35`), SST count — and therefore this term — grows monotonically with bytes written. This is exactly the term #117 bounds for ethrex.
Two blockers were resolved during that review:
1. `client/besu/dbs_cgo.go` claimed `SetCacheIndexAndFilterBlocks` / `SetPartitionFilters` are "not exposed by grocksdb" — **stale**: grocksdb v1.10.8 exposes both (`options_block_based_table.go:122,274`, plus `SetPinL0FilterAndIndexBlocksInCache:131`). The comment is corrected on the #117 fix branch.
2. The concern that routing index/filter blocks through a cache changes the produced DB: it does not — it is a read-path memory-placement option; goldens pin logical KV + state root.
Proposed: mirror the ethrex arrangement — a small shared LRU (ethrex uses 512 MiB) attached to all CFs + `SetCacheIndexAndFilterBlocks(true)`. Do **not** set `SetPinL0FilterAndIndexBlocksInCache` (pinned usage may legally exceed LRU capacity; thousands of uncompacted L0 files would blow past any small cache).
Notes: both writers have survived 100 GB gates at single-digit-GiB peaks, so this term is evidently smaller for them at that scale than it was for ethrex at 350 GB — this is preventive parity, priority accordingly. Validate with a bench run + the goldens per client.
Contributor guide
Research direction
Start with client/besu/dbs_cgo.go:129-131 and client/nethermind/dbs_cgo.go:33-35, then compare the cache setup from the ethrex #117 fix and the exposed grocksdb options in options_block_based_table.go. Apply the shared-cache arrangement to both writers without pinning L0 filter and index blocks, then validate with a bench run and the goldens for each client.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100