QuantEcon / QuantEcon/QuantEcon.py

MAINT: Make the slow marker effective; reconsider wall-clock assertions in the unit suite

Open
#890 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

maintenance
Dominant language
Python
Stars
2.4k
Forks
2.3k
Avg merge
3d 3h
Merged PRs (30d)
3

Description

Problem

pytest.ini defines a slow marker and three tests carry it, but CI runs pytest quantecon with no -m filter, so the marker has no effect anywhere — documentation of an intent that was never wired up.

Two of the three marked tests — test_gridtools.py::test_performance_C and ::test_performance_F — are wall-clock performance assertions, together roughly 15 seconds of the suite. Timing assertions on shared CI runners are a well-known flakiness source, and the project has already had to de-flake timing tests once (#846).

Two things to decide
  1. Should CI deselect slow on the pull-request path and run the full suite on merge to main? Cheap, and makes the marker mean something.
  2. Do wall-clock assertions belong in a correctness suite at all? If the intent is to catch performance regression, a benchmark job comparing against a baseline is the tool; a unit test that fails when a runner is busy is not.
Acceptance criteria
  • A decision recorded on where slow tests run, and CI matches it
  • test_performance_C/test_performance_F converted to relative benchmarks, given meaningful tolerances, or removed with their intent documented
  • The slow marker either affects selection somewhere, or is removed from pytest.ini

From the July 2026 technical-debt audit (AI-assisted; claims verified against 28d4b3b on 2026-07-25).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with pytest.ini, the CI configuration, and quantecon tests test_gridtools.py::test_performance_C and test_performance_F. Determine how the slow marker should affect pull-request and main-branch runs, then resolve the timing-test approach and update selection or marker configuration. Done means the acceptance criteria are met and the suite behavior is documented by tests or configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.