ethereum / ethereum/execution-specs
Consider reducing fixture size for `account_query` (16 GB) and `unchunkified_bytecode` (13 GB)
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 505
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 116
Description
## Summary
Grab the latest artifact like this:
```
gh run download -R ethereum/execution-specs -n fixtures_benchmark_fast -D /mnt/eest_artifacts
```
Two benchmark fixture files are extreme size outliers — 100-1000× larger than all others. This issue documents the specific redundancies driving the size and suggests possible approaches to reduce it.
## Top-10 Benchmark Fixture Sizes
| Rank | Fixture | Size |
|------|---------|------|
| 1 | `account_query.json` | **16 GB** |
| 2 | `unchunkified_bytecode.json` | **13 GB** |
| 3 | `block_full_of_ether_transfers.json` | 123 MB |
| 4 | `block_full_data.json` | 42 MB |
| 5 | `auth_transaction.json` | 19 MB |
| 6 | `block_full_of_precompile.json` | 15 MB |
| 7 | `block_full_of_sha256_hashes.json` | 14 MB |
| 8 | `secp256k1.json` | 13 MB |
| 9 | `kzg_point_evaluation.json` | 13 MB |
| 10 | `modexp.json` | 9.3 MB |
The gap between #2 (13 GB) and #3 (123 MB) is >100×. The remaining fixtures (#3–#10) are proportionate to their workload.
---
## Outlier 1: `account_query.json` (16 GB)
**Test:** [`test_account_query`](https://github.com/ethereum/execution-specs/blob/forks/amsterdam/tests/benchmark/compute/instruction/test_account_query.py#L428) at `tests/benchmark/compute/instruction/test_account_query.py:428`
**Parametrization:** 8 opcodes × 2 `access_warm` × 4 `mem_size` × 5 `code_size` × 2 `value_sent` = 640 theoretical combinations, reduced to ~94 by skip logic (lines 440–452).
**Root cause:** The 8 EXTCODECOPY + `max_code_size` cases (4 `mem_size` × 2 `access_warm`) each generate ~1.8 GB, totalling ~14.4 GB (roughly 96% of the file). Each case deploys ~44K contracts of 24,576 bytes via `CustomSizedContractFactory`.
### Redundancy flags
#### 1. `mem_size=0` with EXTCODECOPY is a no-op copy
At [line 482–489](https://github.com/ethereum/execution-specs/blob/forks/amsterdam/tests/benchmark/compute/instruction/test_account_query.py#L482), the EXTCODECOPY call uses `size=mem_size`. When `mem_size=0`, zero bytes are copied — this benchmarks account access cost only, which is functionally identical to what BALANCE/EXTCODESIZE already measure. It doesn't exercise EXTCODECOPY's actual code-copying behavior. This generates ~3.6 GB (2 warm variants) of fixtures that don't measure what EXTCODECOPY uniquely does.
#### 2. `mem_size` variations produce near-identical fixtures
The 4 `mem_size` values (0, 32, 256, 1024) change gas cost per iteration slightly, resulting in slightly different numbers of max-size contracts deployed. But the pre-state structure is essentially the same — thousands of 24 KB contracts. This creates 4× fixture duplication for the EXTCODECOPY + max_code_size parameter space.
#### 3. `access_warm=True` generates massive access lists
Warm variants include an `AccessList` entry for every target contract (~44K entries), roughly doubling each already-massive fixture. See [lines 549–565](https://github.com/ethereum/execution-specs/blob/forks/amsterdam/tests/benchmark/compute/instruction/test_account_query.py#L549) for the access list generation.
### Possible approaches
- Remove `mem_size=0` for EXTCODECOPY (it doesn't test code copying)
- Reduce `mem_size` to fewer values (e.g., just 0 and 1024 for boundary testing)
- Consider whether `access_warm=True` with `max_code_size` is worth the ~7 GB it adds
---
## Outlier 2: `unchunkified_bytecode.json` (13 GB)
**Test:** [`test_unchunkified_bytecode`](https://github.com/ethereum/execution-specs/blob/forks/amsterdam/tests/benchmark/compute/scenario/test_unchunkified_bytecode.py#L37) at `tests/benchmark/compute/scenario/test_unchunkified_bytecode.py:37`
**Parametrization:** 7 opcodes (CALL, CALLCODE, DELEGATECALL, STATICCALL, EXTCODESIZE, EXTCODEHASH, EXTCODECOPY), all cold access against max-code-size contracts.
**Root cause:** Each opcode deploys its own independent set of ~44K max-size (24,576-byte) contracts. 7 opcodes × ~1.7 GB each ≈ 12 GB.
### Redundancy flag: All 7 opcodes deploy identical contract sets independently
Every test case calls `CustomSizedContractFactory(pre=pre, fork=fork)` with default `contract_size=None` (max). The setup phase is identical across all 7 — the only difference is the opcode used in the execution phase. The contract deployment dominates the fixture size.
The 7 opcodes group into 3 distinct gas-cost categories:
| Category | Opcodes | Cold gas cost |
|----------|---------|---------------|
| Metadata | EXTCODESIZE, EXTCODEHASH | 2,600 |
| Copy | EXTCODECOPY | 2,696 (with 1000-byte copy) |
| Execution | CALL, CALLCODE, DELEGATECALL, STATICCALL | 2,700 |
Within each category, the gas cost is identical, so the benchmarks are effectively duplicates. EXTCODESIZE and EXTCODEHASH produce the same benchmark result. The 4 CALL variants produce the same benchmark result.
### Possible approaches
- Reduce to 3 representative opcodes (one per gas-cost category) instead of 7
- Alternatively, if all 7 are needed for client-specific behavior differences, document the rationale
---
## Context
The remaining large fixtures (#3–#10) were reviewed and their sizes are proportionate to their workloads — no action needed for those.
Contributor guide
Assessment
This issue has not been assessed yet.