ethereum / ethereum/execution-specs

feat(test-fill): compare Engine X post-state against the sibling fixture in the packing execution check

Closed
#3,161 1 comment 0 reactions 1 assignee Claimed by @danceratopz View on GitHub
A-test-fill C-feat stale
Dominant language
Python
Stars
1.2k
Forks
505
Avg merge
2d 8h
Merged PRs (30d)
106

Description

**Prerequisite**: #3160.

## Context

Pre-alloc group packing (#3122) merges Engine X pre-allocation groups and guards against leaked accounts with a fill-time check: every Engine X fixture's `engineNewPayloads` is compared against the test's `blockchain_test_engine` sibling (filled from the test's own pre-allocation with an independent `t8n` execution), scrubbing only the state-root-derived fields (`stateRoot`, `blockHash`, `parentHash`).

## Blind spot

Scrubbing the state root leaves one class of leak invisible: an account introduced by packing whose state a test reads and re-stores with identical gas (for example, storing a value derived from `BALANCE`/`EXTCODEHASH` of the leaked address, where both `SSTORE` transitions cost the same). This changes only stored values, not gas, logs, receipts, or requests, and passes the payload comparison. If the affected slots are not asserted in the test's `post`, the fill-time post check misses it too. The fixture stays self-consistent for consumers, so nothing goes red: the cost is silent semantic dilution of what the test exercises.

## Proposed check

Extend `verify_engine_x_execution` so that, when the payloads match, the Engine X fixture's `postStateDiff` is compared account-by-account against the sibling's full `postState` (a deleted account appears as `null` in the diff and must be absent from the sibling's post-state). Identical execution must produce identical changed accounts, so any mismatch is drift.

Also count and warn loudly when the post-state comparison is skipped (missing `postStateDiff` or missing sibling `postState`, e.g. benchmarks with `include_full_post_state_in_output = False`), so a regression in diff emission cannot silently disable the check again.

## Existing draft

@spencer-tb initially added the check in danceratopz/execution-specs#60, which implements the comparison plus a duplicate-test-id guard in `pack_pre_alloc_groups`. It needs a rebase onto the merged #3122 (the index now stores `GroupIndexEntry` tuples and `verify_engine_x_execution` returns `EngineXCheckResult`, not a summary string) and the skip-count warning above. Without #3160 it silently no-ops, because `postStateDiff` is never present.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.