ethereum / ethereum/state-actor

besu/nethermind: bound RocksDB index+filter blocks via cache_index_and_filter_blocks (parity with ethrex #117)

Open
#129 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
12
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Follow-up from the #117 review.

The besu and nethermind writers leave RocksDB's `cache_index_and_filter_blocks` at its default (`false`), and set **no block cache bound at all** — so every flushed SST's index + bloom-filter blocks are pre-loaded into per-SST table readers, outside any budget, unevictable while the file stays open. With L0 compaction triggers pinned to `MaxInt32` for the whole import (`client/besu/dbs_cgo.go:129-131`, `client/nethermind/dbs_cgo.go:33-35`), SST count — and therefore this term — grows monotonically with bytes written. This is exactly the term #117 bounds for ethrex.

Two blockers were resolved during that review:

1. `client/besu/dbs_cgo.go` claimed `SetCacheIndexAndFilterBlocks` / `SetPartitionFilters` are "not exposed by grocksdb" — **stale**: grocksdb v1.10.8 exposes both (`options_block_based_table.go:122,274`, plus `SetPinL0FilterAndIndexBlocksInCache:131`). The comment is corrected on the #117 fix branch.
2. The concern that routing index/filter blocks through a cache changes the produced DB: it does not — it is a read-path memory-placement option; goldens pin logical KV + state root.

Proposed: mirror the ethrex arrangement — a small shared LRU (ethrex uses 512 MiB) attached to all CFs + `SetCacheIndexAndFilterBlocks(true)`. Do **not** set `SetPinL0FilterAndIndexBlocksInCache` (pinned usage may legally exceed LRU capacity; thousands of uncompacted L0 files would blow past any small cache).

Notes: both writers have survived 100 GB gates at single-digit-GiB peaks, so this term is evidently smaller for them at that scale than it was for ethrex at 350 GB — this is preventive parity, priority accordingly. Validate with a bench run + the goldens per client.

Contributor guide

Open the contributing guide

Research direction

Start with client/besu/dbs_cgo.go:129-131 and client/nethermind/dbs_cgo.go:33-35, then compare the cache setup from the ethrex #117 fix and the exposed grocksdb options in options_block_based_table.go. Apply the shared-cache arrangement to both writers without pinning L0 filter and index blocks, then validate with a bench run and the goldens for each client.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.