ethereum / ethereum/execution-specs
Automate cold-access transaction splitting for stateful benchmarks
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 505
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 116
Description
The bloatnet benchmark tests (`test_multi_opcode.py`, `test_single_opcode.py`) currently implement manual transaction splitting logic that duplicates functionality in `BenchmarkTest.split_transaction`. However, the existing `split_transaction` method creates **identical transactions**, which defeats the purpose of cold-access benchmarks when `tx_gas_limit` caps (like Fusaka's 16M limit) require multiple transactions.
We could consider extending `BenchmarkTest.split_transaction` to support **offset-aware splitting** where each transaction receives unique calldata that tells the attack bytecode where to start iterating.
### Background
See PR discussion: https://github.com/ethereum/execution-specs/pull/1962#issuecomment-3706540450
**Current behavior (identical txs)**:
```
Tx 1: Access addresses with salts 0-999 -> all COLD (2600 gas each)
Tx 2: Access addresses with salts 0-999 -> all WARM (100 gas each) <- Wrong!
Tx 3: Access addresses with salts 0-999 -> all WARM (100 gas each) <- Wrong!
```
**Desired behavior (offset-based txs)**:
```
Tx 1: Access addresses with salts 0-999 -> all COLD (2600 gas each)
Tx 2: Access addresses with salts 1000-1999 -> all COLD <- Correct!
Tx 3: Access addresses with salts 2000-2999 -> all COLD <- Correct!
```
### Current state
[`BenchmarkTest.split_transaction`](https://github.com/ethereum/execution-specs/blob/b3543e94d12288e994fc1adea606c1a417db4a9f/packages/testing/src/execution_testing/specs/benchmark.py#L374-L402).
- **Handles**: gas limit calculation, transaction copying, nonce increment.
- **Does NOT handle**: per-transaction calldata with offset information.
### Bloatnet Test Patterns
Two patterns exist in bloatnet tests:
1. **CREATE2-based tests** (`test_bloatnet_balance_extcodesize`, etc.):
- Generate addresses using `keccak256(0xFF | factory | salt | init_code_hash)`
- Salt starts at 0, increments each iteration
- Bytecode: `Op.MSTORE(32, Op.ADD(Op.MLOAD(32), 1))` to increment
2. **ERC20-based tests** (`test_sload_empty_erc20_balanceof`, etc.):
- Call pre-deployed ERC20 contracts with varying address/spender
- Counter starts at N, decrements each iteration
- Each counter value maps to a different storage slot inside ERC20
Both patterns share the same problem: the iteration starting point is hardcoded in bytecode, not read from calldata.
Contributor guide
Assessment
This issue has not been assessed yet.