python / python/mypy

mypyc incremental failures on master

Open
#13,572 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic-incremental topic-mypyc topic-tests
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

We've started seeing non deterministic failures of mypyc incremental tests in CI.

The first test failure on master was on https://github.com/python/mypy/commit/840a310f0b6c4a4bd7edfb141c8d1731ba4dd027 (succeeded on retry).

The earliest test failure I've observed was on https://github.com/python/mypy/pull/13557 when it was based on d9bdd6d96.

No known changes to Python version or dependencies. The underlying Github runner image did change, but nothing stands out in the changelog and it doesn't seem likely to cause failures.

To my knowledge, no one has been able to reproduce failures locally.

When running locally on Linux, I get deterministic failures on:

FAILED mypyc/test/test_run.py::TestRun::run-loops.test::testForIterable
FAILED mypyc/test/test_run.py::TestRun::run-generators.test::testYieldThrow

But these deterministically reproduce and are unrelated to incremental, and reproduce on master hundreds of commits ago, so seem unrelated.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare the CI failures around commits 840a310f0b6c4a4bd7edfb141c8d1731ba4dd027 and d9bdd6d96, then inspect the mypyc incremental test setup. The locally reproducible cases are in mypyc/test/test_run.py, but the issue says they are unrelated. Done means reproducing and isolating the nondeterministic incremental failure and identifying a reliable corrective change.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.