ethereum / ethereum/execution-specs
First Class Ported Static Tests Tracker
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 505
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 116
Description
### Context
The 2,095 machine-ported tests in `tests/ported_static/` started as flat `stXxx/` directories with a blanket `valid_from("Cancun")` and, on Amsterdam, a 153-entry skip list. The end goal of this tracker is to make them first class native tests, fork-organized, minimally floored, fork-robust, and ultimately merged into the `tests//` structure properly.
**Aim:** the overarching aim is to make it as though it looks like they never existed, move all the ported static tests into `tests/` and fill them all as low as there floor allows for. Ideally from `Frontier` for most of them.
### Phases
Note all PRs rely on each parents merging, i.e one after the other, they are built upon each other.
#### Removing The Ported Static Skip List
- [x] Part 1 — enhance & un-skip Amsterdam ported static tests: #3215
- [ ] Part 2 — un-skip the remaining 130, empty and remove the skip list + conftest machinery: #3264
#### Restructuring
- [ ] Fork-folder reorg, pure moves into `tests/ported_static///` (subject fork, decoupled from `valid_from`; layout enforced by conftest; fill-verified inert): spencer-tb/execution-specs#33
- [ ] Empirical `valid_from` floors, `protected=fork.supports_protected_txs()` for 1,716 tests + 1,690 markers lowered (1,128 now floor at Frontier; 119,610 full-range cases green; minimality audited — 392 floors provably equal their newest-used-opcode bound, 0 defects): spencer-tb/execution-specs#34
- [ ] Gas-limit hygiene — 226 non-load-bearing hardcoded tx gas limits removed (individually gated, 91 load-bearing candidates rolled back): spencer-tb/execution-specs#35
These 3 PRs will be retargeted to `forks/amsterdam` sequentially as their bases merge.
**Snapshot floor parity** (required before archiving `ethereum/tests` + `ethereum/legacytests`, see #2556)
An audit of every ported test's floor against the per-fork `post` sections of the frozen snapshots (`ethereum/tests` HEAD tarball + `legacytests/Constantinople` + `legacytests/Cancun`) shows 1,403 tests at parity or better, and **670 tests where the snapshots covered lower forks than we currently do** — archiving today would lose that coverage. Largest classes: Cancun-floored fragile tests whose originals reached Byzantium/Frontier/Istanbul, and TangerineWhistle/SpuriousDragon-floored tests whose originals expressed *per-fork* expectations (reaching parity there needs fork-conditional posts, not just lower markers).
- [ ] Close the 670 floor-parity regressions (per-file targets in the audit CSV)
- [ ] Re-run the parity audit clean before any archive decision
**Remaining enhancement queue** (floors are code-bound, not subject-bound; the `enhance-ported-test` recipes apply)
- [ ] 48 gas-snapshot tests → CodeGasMeasure rewrites
- [ ] 63 + 91 fragile tests (hand triage + load-bearing-limit files)
- [ ] Documented honest floors, no change planned: 98 TangerineWhistle (forward-all gas shape), 47 SpuriousDragon (EIP-161 fundamentals), 18 Berlin (EIP-2929 metadata), 19 London (EIP-3529 refund semantics)
**Endgame**
- [ ] Rename suites/files to native conventions (`stEIP3855_push0` → `eip3855_push0`, meaningful names for the random/fuzzed blobs where feasible)
- [ ] Merge into the native `tests//` structure and dissolve `tests/ported_static/` (reference-spec wiring, `ported_from` markers retained for `ethereum/tests` traceability)
Contributor guide
Assessment
This issue has not been assessed yet.