ethereum / ethereum/execution-specs
feat(test-fill): compare Engine X post-state against the sibling fixture in the packing execution check
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 505
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 106
Description
**Prerequisite**: #3160.
## Context
Pre-alloc group packing (#3122) merges Engine X pre-allocation groups and guards against leaked accounts with a fill-time check: every Engine X fixture's `engineNewPayloads` is compared against the test's `blockchain_test_engine` sibling (filled from the test's own pre-allocation with an independent `t8n` execution), scrubbing only the state-root-derived fields (`stateRoot`, `blockHash`, `parentHash`).
## Blind spot
Scrubbing the state root leaves one class of leak invisible: an account introduced by packing whose state a test reads and re-stores with identical gas (for example, storing a value derived from `BALANCE`/`EXTCODEHASH` of the leaked address, where both `SSTORE` transitions cost the same). This changes only stored values, not gas, logs, receipts, or requests, and passes the payload comparison. If the affected slots are not asserted in the test's `post`, the fill-time post check misses it too. The fixture stays self-consistent for consumers, so nothing goes red: the cost is silent semantic dilution of what the test exercises.
## Proposed check
Extend `verify_engine_x_execution` so that, when the payloads match, the Engine X fixture's `postStateDiff` is compared account-by-account against the sibling's full `postState` (a deleted account appears as `null` in the diff and must be absent from the sibling's post-state). Identical execution must produce identical changed accounts, so any mismatch is drift.
Also count and warn loudly when the post-state comparison is skipped (missing `postStateDiff` or missing sibling `postState`, e.g. benchmarks with `include_full_post_state_in_output = False`), so a regression in diff emission cannot silently disable the check again.
## Existing draft
@spencer-tb initially added the check in danceratopz/execution-specs#60, which implements the comparison plus a duplicate-test-id guard in `pack_pre_alloc_groups`. It needs a rebase onto the merged #3122 (the index now stores `GroupIndexEntry` tuples and `verify_engine_x_execution` returns `EngineXCheckResult`, not a summary string) and the skip-count warning above. Without #3160 it silently no-ops, because `postStateDiff` is never present.
Contributor guide
Assessment
This issue has not been assessed yet.