ethereum / ethereum/execution-specs

perf(test-fill): every engine_x t8n call copies and re-merklizes the whole merged pre-alloc group

Open
#3,403 0 comments 0 reactions 2 assignees Claimed by @LouisTsai-Csie View on GitHub
A-test-fill C-perf
Dominant language
Python
Stars
1.2k
Forks
505
Avg merge
2d 14h
Merged PRs (30d)
116

Description

Dan says: I think this is a low-ish hanging fruit for optimizing regular release builds and benchmark builds alike. My main idea was to have a daemon that gets the genesis pre-alloc once and holds it for all tests in that group. Claude breaks it down further and sees that as a reasonable additional optimization, but not the first thing to implement.

### Problem

Engine_x fixtures share one merged pre-state per pre-alloc group. When phase 2 fills the engine_x format for a test, the filler swaps the test's own pre-state for the merged group pre-state (`packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py:1709`) and calls t8n with it.

Since #2924, t8n runs in process, and every call does work proportional to the size of the whole group, not to the handful of accounts the test touches:

1. It deep-copies the whole merged alloc (`packages/testing/src/execution_testing/evm_tools/t8n/__init__.py:228`).
2. It rebuilds the code cache on the copy.
3. It rebuilds the spec `State` from the alloc.
4. It re-merklizes every account from scratch to compute the state root.
5. The filler then compares every account in the input and output states to compute the post-state diff.

Each test only touches a handful of accounts but pays for all of them. And it pays per block, not per test, because block N's input is the full merged post-state.

### Measured cost

Per t8n call, against real merged group files from previous fill runs (CPython 3.12):

| # | Stage | 23k accounts (41 MB) | 466k accounts (388 MB) |
|---|-------|---------------------:|------------------------:|
| 1 | deep copy of the merged alloc | 0.79 s (41 %) | 14.0 s (33 %) |
| 2 | code-cache rebuild on the copy | 0.11 s (6 %) | 0.7 s (2 %) |
| 3 | rebuild spec `State` from the alloc | 0.16 s (8 %) | 4.1 s (10 %) |
| 4 | full state-root merklization | 0.78 s (41 %) | 21.3 s (50 %) |
| 5 | post-state diff (compare every account) | 0.08 s (4 %) | 2.2 s (5 %) |
| | total per call | 1.92 s | 42.3 s |

Summed over real corpora:

| Corpus | Groups | engine_x tests | Overhead estimate |
|--------|-------:|---------------:|-------------------|
| `tests/benchmark/compute`, gas=1 (fast) | 1 big | 1,575 | ~50 CPU-min |
| benchmark, full gas values | 1 big | 6,587 | ~77 CPU-h |
| mainnet full release | 6,553 | 65,926 | ~7 to 9 CPU-h |

The per-worker group load path adds to this: parse and validate, a full genesis re-merklization, then a redundant `model_dump` plus a second `model_validate`. That is 1.9 s for the 41 MB group and about 29 s for the 388 MB one, per worker.

Memory: the 388 MB group parses to about 1.8 GB of heap, and the deep copy temporarily doubles that (3.4 GB RSS measured).

### Proposed fix

Three in-process changes that compose. Each is a separate PR, in order:

- **opt-0: load-path fixes and code-hash caching.** Compute the group genesis at merge time, drop the redundant dump/validate round trip on load, and cache code hashes instead of re-hashing account code on every read. Small and immediate.
- **opt-1: frozen shared pre-state, diff-based post-state.** Freeze the group pre-state once per worker and never copy it. t8n reads the frozen alloc and returns a diff; the post-state becomes an overlay over the frozen pre-state. Kills stages 1, 2 and 5 (roughly 40 % of the overhead).
- **opt-2: per-group cached trie with incremental roots.** Keep the group trie alive and update the root from the diff instead of re-merklizing every account (roughly 60 % of the overhead). This one carries a correctness argument: it ships with an opt-in assertion mode that computes both roots and checks they match, run over a full corpus at least once before merging.

Together these take the per-test overhead on the largest group from 42.3 s to an estimated under 0.3 s, which makes benchmark fills EVM-bound.

A t8n daemon (state shared across workers) is deferred, not rejected. opt-1 keeps the t8n contract daemon-shaped (read a frozen pre-state, return a diff), so it can be lifted behind a socket later if group sizes outgrow per-worker memory.

### Verification

Every opt proves fixture transparency with `hasher compare` before and after on a representative corpus, plus timing evidence from the fast benchmark fill (`tests/benchmark/compute` at gas=1).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.