ethereum / ethereum/execution-specs
Enable t8n caching for state_test on Paris/Shanghai
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 505
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 116
Description
## Background
[PR #2084](https://github.com/ethereum/execution-specs/pull/2084) introduced t8n output caching so that fixture formats sharing the same `transition_tool_cache_key` can reuse t8n results instead of calling the transition tool again.
## Opportunity
The `state_test` fixture format does not participate in t8n output caching (`transition_tool_cache_key = ""`). When a `state_test` and `blockchain_test` are filled for the same test function, the `state_test` makes its own t8n call even when the output is identical to an already-cached `blockchain_test` result.
Current execution order and t8n calls per StateTest:
| Format | t8n calls | Cache role |
| ------------------------------------------ | --------- | ---------------------- |
| `blockchain_test_from_state_test` | 1 | populates cache |
| `blockchain_test_engine_from_state_test` | 0 | cache hit |
| `state_test` | 1 | **no caching** |
| `blockchain_test_engine_x_from_state_test` | 1 | no caching (by design) |
## Why `state_test` can share cache with `blockchain_test` for Paris and Shanghai only
The t8n output for `state_test` and `blockchain_test` is identical when two conditions hold:
1. **Reward = 0** - Both formats pass reward=0 to t8n. True for all post-Merge forks (Paris onward), false for pre-Merge forks where difficulty handling differs.
2. **`pre_allocation()` == `pre_allocation_blockchain()` == `{}`** - The `state_test` format uses `pre_allocation()` while `blockchain_test` uses `pre_allocation_blockchain()`. For Paris and Shanghai, both return `{}` (no system contracts). Starting from Cancun, `pre_allocation_blockchain()` adds system contracts (beacon root via EIP-4788, then history storage, consolidation requests in Prague+) that `pre_allocation()` does not, so the t8n inputs diverge.
## Proposed implementation
### 1. Add `transition_tool_cache_forks` ClassVar to `BaseFixture`
**File:** `packages/testing/src/execution_testing/fixtures/base.py`
Add alongside `transition_tool_cache_key` (line 84):
```python
transition_tool_cache_forks: ClassVar[Sequence[type] | None] = None
```
- `None` (default): cache key is valid for all forks (existing behavior).
- Sequence of Fork classes: cache key is only valid for those forks.
### 2. Set both ClassVars on `StateFixture`
**File:** `packages/testing/src/execution_testing/fixtures/state.py`
```python
from execution_testing.forks import Fork, Paris, Shanghai
class StateFixture(BaseFixture):
...
transition_tool_cache_key: ClassVar[str] = "blockchain_test"
transition_tool_cache_forks: ClassVar[Sequence[type] | None] = (Paris, Shanghai)
```
### 3. Propagate `transition_tool_cache_forks` as a pytest marker
**File:** `packages/testing/src/execution_testing/cli/pytest_commands/plugins/shared/helpers.py`
In `labeled_format_parameter_set()`, after the existing `transition_tool_cache_key` marker block (~L49-57), add:
```python
transition_tool_cache_forks = getattr(
format_with_or_without_label, "transition_tool_cache_forks", None
)
if transition_tool_cache_forks is not None:
marks.append(
pytest.mark.transition_tool_cache_forks(transition_tool_cache_forks),
)
```
### 4. Fork-aware check in `get_t8n_cache_key()`
**File:** `packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py`
Modify `get_t8n_cache_key()` (L1201-1208) to check for a fork restriction:
```python
def get_t8n_cache_key(request: pytest.FixtureRequest) -> str | None:
"""Get the cache key to be used for the current test, if any."""
mark = request.node.get_closest_marker("transition_tool_cache_key")
if mark is None or len(mark.args) != 1:
return None
# Check fork restriction if present.
forks_mark = request.node.get_closest_marker("transition_tool_cache_forks")
if forks_mark is not None and len(forks_mark.args) == 1:
allowed_forks = forks_mark.args[0]
fork = request.node.callspec.params.get("fork")
if fork is None or fork not in allowed_forks:
return None
return f"{strip_fixture_format_from_node(request.node)}-{mark.args[0]}"
```
### 5. Add tests
**File:** `packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/tests/test_t8n_cache.py`
Add a `TestForkConditionalCaching` class testing:
- `state_test` on Paris/Shanghai - returns cache key (matches `blockchain_test`)
- `state_test` on Cancun/other forks - returns `None` (no caching)
- `blockchain_test` (no fork restriction) - returns cache key for all forks
### Sorting impact
No changes needed. `state_test` items with `transition_tool_cache_key` set will sort as cacheable (grouped with the blockchain formats). For non-Paris/Shanghai forks, `get_t8n_cache_key()` returns `None` at runtime - `remove_cache()` is called, but this is harmless because it runs AFTER the blockchain pair has already consumed the cache (`state_test` sorts last alphabetically among cacheable formats: `blockchain_test` < `blockchain_test_engine` < `state_test`).
## Expected savings
Per-test t8n calls on Paris/Shanghai for default `fill` options (no EngineX):
| Before #2084 | After #2084 | After this issue |
| ------------ | ----------- | ---------------- |
| 3 calls | 2 calls | 1 call |
## Verification plan
1. `uvx tox -e static` - type checking and linting pass
2. `uvx tox -e tests_pytest_py3,tests_benchmark_pytest_py3` - unit tests pass
3. Manual check: fill exactly one `blockchain_test` and one `state_test` for Shanghai and verify the cache summary shows a hit:
```bash
uv run fill \
"tests/berlin/eip2929_gas_cost_increases/test_call.py::test_call_insufficient_balance[fork_Shanghai-blockchain_test_from_state_test]" \
"tests/berlin/eip2929_gas_cost_increases/test_call.py::test_call_insufficient_balance[fork_Shanghai-state_test]"
```
Expect the terminal summary to report `T8n cache: 100% hit rate (1/1 tests expected), 1 t8n calls saved`.
Contributor guide
Assessment
This issue has not been assessed yet.