ethereum / ethereum/execution-specs

chore(testing): delete or revive test_count_opcodes.py (never runnable since introduction)

Open
#3,326 1 comment 0 reactions 0 assignees View on GitHub
A-test-tests C-chore
Dominant language
Python
Stars
1.2k
Forks
505
Avg merge
2d 14h
Merged PRs (30d)
116

Description

### Context

`test_count_opcodes.py` (now at
`packages/testing/src/execution_testing/evm_tools/tests/`, previously
`tests/evm_tools/`) was added in `38e45e58d` ("Add flag to count opcodes during
execution", 2025-07-29) alongside the opcode-count tracer. It has never run,
in CI or in any default local invocation, because three independent blockers
have been in place since the day it was added:

1. Never collected. The test is marked `@pytest.mark.evm_tools`, and the tox
config of the day ran `pytest -m "not slow and not evm_tools"`. No tox
environment and no CI workflow ever selected the marker. The Justfile era
replaced marker deselection with explicit `--ignore` flags, which #3307
carries forward into two targets (`test-tests`, `test-tests-pypy`).
2. Fixture out of scope. The same commit added the `root_relative` fixture to
`tests/json_infra/conftest.py` (now `tests/json_loader/conftest.py`), but
the test lived in the sibling `tests/evm_tools/` tree with no conftest.
Pytest scopes conftest fixtures to their directory tree, so collection
errors with `fixture 'root_relative' not found`. Reproduced on the #3307
branch.
3. Data never existed. `root_relative("fixtures/evm_tools_testdata/t8n/fixtures/testdata/2")`
resolves under the json loader tree, where `fixtures/` is gitignored
download-time content, and no download step has ever provided
`evm_tools_testdata`. The path shape matches go-ethereum's
`cmd/evm/testdata` layout, so the test appears written against locally
copied geth testdata.

A consequence worth stating: the opcode-count tracer
(`evm_tools/t8n/evm_trace/count.py`) has never had test coverage. This was its
only test.

### Options

1. Delete the test (and the two `--ignore` flags in the Justfile). There is no
previously green state to restore; deletion loses nothing that ever ran.
2. Rewrite it against in-repo inputs. The `t8n_build/` fixtures added by #3307
(alloc/env/txs for a real Frontier transition) are almost exactly the input
this test needs; rewriting against them would give `count.py` its first
real coverage and remove both `--ignore` flags. If the fixtures move into
the testing package (suggested in the #3307 review), the test and its data
would sit side by side.

Option 2 is preferable if anyone uses `--opcode.count`; otherwise option 1.

### Related

- #3307 (moved the test; entrenched the `--ignore` in two targets)
- `38e45e58d` (introduced the test, the marker exclusion, and the misplaced
fixture in one commit)

Contributor guide

Open the contributing guide

Research direction

Start with packages/testing/src/execution_testing/evm_tools/tests/test_count_opcodes.py, evm_tools/t8n/evm_trace/count.py, and the two Justfile targets carrying --ignore flags. Review the t8n_build fixtures from #3307 and the test's historical fixture path, then confirm with maintainers whether deletion or revival is intended. Done means the dead test and exclusions are resolved, with coverage for count.py if the test is revived.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.