ethereum / ethereum/execution-specs

Stateful Tests CI Verification

Open
#2,423 1 comment 0 reactions 1 assignee Claimed by @LouisTsai-Csie View on GitHub
stale
Dominant language
Python
Stars
1.2k
Forks
505
Avg merge
2d 14h
Merged PRs (30d)
116

Description

- **Phase 1 – Basic filling in CI (minimum bar)**
- Add CI jobs that fill stateful benchmark tests and fail if filling breaks
- Focus first on benchmarks without stub accounts to reduce complexity
- For stub accounts, use `eth_getCode` RPC + `evm_bytes` to get the stub bytecode into Python

- **Phase 2 – Hive execution integration**
- Use `execute hive` in CI so we can actually execute them, not just fill

- **Phase 3 – Full workflow automation with Kamil's tooling**
- Integrate Kamil's gas-benchmarks payload generation tool into CI
- Reproduce his full manual workflow: generate payloads, run benchmarks, validate results
- Aim for end-to-end automated verification (generation → execution → validation) to catch bugs early and avoid regressions

Note: Phase 3 seems complex, therefore Phase 2 could be our aim for the following weeks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.