ethereum / ethereum/execution-specs

Decouple Python 3.11 compatibility verification from the py3 CI job

Open
#2,316 6 comments 2 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
1.2k
Forks
505
Avg merge
2d 14h
Merged PRs (30d)
116

Description

## Context

The `py3` CI job currently serves two purposes:

1. **Verify the specs are correct** by filling all (non-slow) tests against EELS with coverage
2. **Verify 3.11 compatibility** — it's pinned to `python-version: "3.11"` in the workflow

If py3 moves to Python 3.14 (PR #2310) for the coverage performance gains discussed in #2314, we lose the only runtime verification that the `ethereum` package works on 3.11 — the minimum version declared in `pyproject.toml` (`requires-python = ">=3.11"`).

## Current state: what actually tests 3.11?

Every job in `test.yaml` that uses CPython is hardcoded to 3.11:

| Job | Python | What it does | Runtime coverage of `ethereum.*` |
| --- | --- | --- | --- |
| `static` | 3.11 | codespell, ruff, mypy, ethereum-spec-lint | No (static analysis only) |
| `py3` | 3.11 | `fill` all tests with `--cov` | **Full** — exercises every fork module |
| `json_infra` | 3.11 | Run spec against released test fixtures | **Substantial** — runs real transactions through fork code |
| `tests_pytest_py3` | 3.11 | Testing package unit tests | Partial — tests the test framework, not the spec itself |
| `pypy3` | pypy3.11 | `fill` all tests (no coverage) | Full — but PyPy, not CPython |

**Only `py3` and `json_infra` exercise the spec code on CPython 3.11.** If `py3` moves to 3.14, `json_infra` becomes the sole 3.11 runtime check — but it only runs against released fixture files, not the full test suite.

## What does "3.11 compatibility" actually mean here?

The risk of accidentally breaking 3.11 compatibility is introducing syntax or stdlib usage that requires 3.12+. In practice this means:

- **Syntax**: f-string nesting (PEP 701), `type` statement (PEP 695), `*` in type params — all 3.12+ only
- **Stdlib**: `pathlib.Path.walk()` (3.12+), `itertools.batched()` (3.12+), `typing` features that moved in 3.12

The spec codebase uses none of these today, and the WET architecture (each fork is a complete copy) means changes are localized.

## Proposal: lightweight 3.11 verification without the py3 job

### Option A: Static enforcement via ruff's `target-version`

Add to `pyproject.toml`:

```toml
[tool.ruff]
target-version = "py311"
```

This makes ruff flag any 3.12+-only syntax or patterns during the existing `static` check (which already runs `ruff check`). This catches the most common class of breakage — new syntax — at zero runtime cost.

**Limitation**: Doesn't catch stdlib usage like `pathlib.Path.walk()` — ruff doesn't lint for that. mypy _could_ in principle, but it doesn't have a `target-version` for stdlib availability.

### Option B: Add `pyupgrade --py311-plus` to static checks

`pyupgrade` rewrites code to use modern syntax. Running it in check mode (`--check`) would flag any 3.12+-only constructs. However, this overlaps heavily with ruff's `UP` rules and adds a new dependency.

### Option C: Run `json_infra` on 3.11, keep it as the 3.11 canary

`json_infra` already exercises the full spec pipeline (parsing fixtures → running state transitions → verifying results). If `py3` moves to 3.14, keeping `json_infra` pinned at 3.11 provides real runtime verification that the spec code loads and executes correctly on 3.11.

This is already the case today — no change needed. The question is whether `json_infra`'s coverage is sufficient. It runs against released fixtures, so it exercises all mainnet forks plus the latest dev fork, but doesn't cover every test case (it's fixture-driven, not parametrized like `fill`).

### Option D: Dedicated minimal 3.11 smoke test

Add a lightweight job that imports every fork module and runs a small subset of tests on 3.11:

```yaml
py311-compat:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-python@...
with:
python-version: "3.11"
- run: tox -e py3 -- tests/homestead tests/osaka/eip7934_block_rlp_limit -x --no-cov -q
```

This would run in ~1-2 minutes and catch import-time failures.

## Recommendation

**Option A (ruff target-version) + Option C (keep json_infra on 3.11)** is the lightest approach:

- ruff catches syntax-level incompatibilities statically, for free
- `json_infra` on 3.11 provides runtime verification via real fixture execution
- No new CI jobs, no new dependencies, no extra runtime cost
- `py3` is free to move to 3.14 for coverage performance

If stronger guarantees are desired, Option D adds a cheap runtime smoke test, but `json_infra` likely already covers this.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.