erigontech / erigontech/erigon

db/state: experiment — use .bt index instead of .vi/.efi accessors (RPC perf + size)

Open
#22,472 5 comments 0 reactions 1 assignee Claimed by @sudeepdino008 View on GitHub
ErigonDB headliner
Dominant language
Go
Stars
3.6k
Forks
1.5k
Avg merge
1d 16h
Merged PRs (30d)
455

Description

## Summary

Experiment: use a B-tree index (`.bt`) as the accessor for history (`.v`) and inverted-index (`.ef`) files, instead of the current recsplit accessors (`.vi` / `.efi`). Measure the trade-off in **RPC performance** and **on-disk size**.

## Motivation

`.bt` indexes are already used as an alternative accessor for domain `.kv` files. Extending the same approach to history/idx accessors could change both lookup latency and index size. The accessor indexes are non-trivial on disk today:

| chain | `.vi` | `.efi` |
|---|--:|--:|
| gnosis v36 | 22.3 GB | 11.2 GB |
| ethmainnet v36 | 418.4 GB | 45.8 GB |

## Experiment

Compare `.bt`-based accessors vs. the current `.vi`/`.efi` recsplit accessors on the same datadir, measuring:

- **RPC performance** — latency/throughput across representative RPC methods (state reads, history/as-of queries, log/trace queries).
- **Index size** — `.bt` size vs `.vi`+`.efi` size, per domain and total.

## Deliverables

- Build both accessor variants for the same snapshots.
- RPC benchmark results (before/after) for gnosis and ethmainnet.
- Size comparison table.
- Recommendation on whether to switch.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.