pydata / pydata/xarray

Disable explicit mypy tests?

Open
#10,934 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CI
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What is your issue?

The explicit mypy tests introduced in https://github.com/pydata/xarray/pull/10038 seem to be quite fragile, e.g., they are currently broken at head and I know have been broken at head previously at least once. In neither case was there a real regression.

I see at least three issues with them:

  1. They seem to be particularly prone to version skew issues where they fail on GitHub but pass locally due to slight differences in environments. I think the current failure is due to mis-matched NumPy versions. (This might be improved when we switch to pixi.)
  2. They are really slow. Testing ~40 specific errors takes ~9 minutes on a single GitHub CI worker, which is not a great use of CI resources and clearly not scalable. Currently we use two CI workers just to test typing for the pipe method!
  3. They are inherently very fragile, checking for exact messages printed by mypy. Pretty much any package update causes error messages to change, which means these tests break, whether or not there is a real underlying issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the explicit mypy tests introduced in PR #10038 and the GitHub CI jobs that run them. Review their current failures, runtime, and typing coverage; done means the fragile explicit tests are disabled without obscuring the remaining typing checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, testing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.