mpfaffenberger / mpfaffenberger/code_puppy

Test infra: conftest async runner conflicts with pytest-asyncio auto mode; 64 metric-named *_coverage files; --cov forced on every run

Open
#430 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
814
Forks
278
Avg merge
2d 5h
Merged PRs (30d)
76

Description

Problem

Test-infrastructure findings from reviewing tests/ (402 tracked files):

  1. Duplicate/conflicting async test runners. pyproject.toml configures pytest-asyncio (asyncio_mode = "auto", and pytest-asyncio>=0.23.1 is a dev dependency), yet tests/conftest.py:138-153 also implements a hand-rolled pytest_pyfunc_call hook whose docstring claims it exists "without pytest-asyncio":

    def pytest_pyfunc_call(pyfuncitem: pytest.Item) -> bool | None:
        """Enable running `async def` tests without external plugins. ..."""
        if inspect.iscoroutinefunction(test_func):
            ...
            asyncio.run(test_func(**kwargs))
            return True
    

    Two mechanisms now compete to run every coroutine test; which one wins depends on plugin hook ordering. The hand-rolled one also bypasses pytest-asyncio's event-loop fixtures, so any test using event_loop/asyncio fixtures behaves differently from what the config promises. Pick one (drop the conftest hook, keep asyncio_mode = "auto").

  2. Coverage-driven test file proliferation. 64 test files are named *_coverage.py / *_full_coverage.py, often three or four files targeting the same module (e.g., test_session_storage.py + _coverage + _edge_cases + _extended = 1,715 lines for a 338-line module; test_terminal_utils* = 2,396 lines for a 512-line module). Naming tests after the metric instead of the behavior guarantees overlap and makes it impossible to find "the test for X". Merge per module and name by behavior.

  3. addopts = "--cov=code_puppy --cov-report=term-missing" forces coverage instrumentation on every pytest invocation, including single-test debugging runs, slowing the inner loop substantially and breaking pytest --pdb line numbers in some setups. Move coverage flags to CI invocation or a make coverage target.

  4. pytest_sessionfinish (conftest.py:156-183) shells out to git status after every run and carries a commented-out file-deletion block plus a no-op line os.path.join(session.config.invocation_dir, rel_path) (result discarded, line 175). Delete the dead code; the untracked-file warning belongs in lefthook/CI, not in every developer test run.

Suggested fix

Remove the conftest pytest_pyfunc_call hook, fold *_coverage files into their per-module canonical test file, move --cov out of addopts, and clean up pytest_sessionfinish.

Filed by Zen Reviewer C (code-puppy-60635a)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with pyproject.toml and tests/conftest.py:138-183, then inventory the 64 *_coverage.py and *_full_coverage.py files named in tests/. Run the relevant pytest commands before and after the cleanup; done means async tests use the configured runner, coverage is opt-in, session cleanup is removed, and behavior-focused tests remain organized per module.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
developer-experience, testing-qa, tooling
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.