ethereum / ethereum/state-actor

ethrex: SstFileWriter/IngestExternalFile bulk path — the ~1.3× write-amp endgame

Open
#131 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
12
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Follow-up designed during the ethrex writer performance work (`perf/ethrex-writer` branch); deliberately deferred as its own project.

## The endgame: SstFileWriter + IngestExternalFile for the sorted CFs

The current (post-perf-series) write path still pays memtable→L0-flush→compaction for every byte. The measured floor with the 2 GiB level-base is ~4× physical writes vs final size at 40 GB scale (~4.7× at 150 GB). The canonical RocksDB bulk-load path — build SSTs directly with `SstFileWriter` and `IngestExternalFile` them at the bottom level — reaches ~1.3×, eliminates Close-time compaction for ingested CFs, and cuts the transient-disk peak.

### What the ordering analysis established (verified, see the perf-branch review)

- `account_flatkeyvalue`: keys emitted **strictly ascending** by construction (Stage C strict-seq; proof in the analysis) → directly writable to SST files.
- `storage_flatkeyvalue`: ascending within and across accounts in autofill; **spec runs break it** (Phase 0 workers write unordered first) → per-CF all-or-nothing means spec runs either keep the memtable path for this CF or Phase 0 routes through the same global sort.
- Trie-node CFs: the right-spine builder emits post-order (root LAST, at the empty key) — leaf-node rows alone are ascending, internal nodes (~⅓ of rows, ~40-65 GB at 150 GB) need a dedicated streamsort pass, then a 2-way merge of the two ascending streams at ingest. The Builder can tag row kind to split the streams.
- `account_codes` is EXCLUDED: blob files cannot be produced by SstFileWriter (representativeness constraint).
- Files must be self-cut at `target_file_size_base` (256 MiB) — one giant SST is not representative.
- grocksdb exposes the full API (`sst_file_writer.go`, `IngestExternalFile*`).

### Acceptance

Golden root + genesis-dump tests unchanged; e2e boot green; write-amp measured ≤1.5× on the 40 GB protocol; no regression in the CALIBRATION cross-client wall table.

Contributor guide

Open the contributing guide

Research direction

Review the perf/ethrex-writer branch analysis and grocksdb's sst_file_writer.go and IngestExternalFile* APIs first. Trace ordering for each sorted column family, including the trie-node stream split and account_codes exclusion. Done means self-cut 256 MiB SSTs, unchanged golden/genesis and e2e results, write amplification at or below 1.5× on 40 GB, and no CALIBRATION regression.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.