[ENH] More type-stubs in the mypy pre-commit environment?
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
Following on from #11640 (specifically https://github.com/rapidsai/cudf/pull/11640#discussion_r961551760) a question arises as to how complete we should make the mypy pre-commit environment in terms of supported typestubs.
Status quo:
The version of mypy installed via pre-commit only depends on the stdlib typeshed stubs (and after #11640 `types-cachetools`). We set `ignore_missing_imports = True` which means that mypy doesn't complain if it sees an import for something (say numpy) that it can't find. Consequently, any non-importable modules are typed as `Any` (as are all objects, methods, functions, etc... from that module); this is a type that always satisfies any type constraint.
In the development environment, all cudf modules _are_ importable, and so a type-checking run using that environment will deduce narrower types for many function calls in cudf. Many of these currently do not type-check, being of the following form: https://mypy-play.net/?mypy=latest&python=3.10&gist=8d3ba6046bb8ca39c6d6b71b442b432c
```python
from typing import Any
def foo(x: Any) -> Any:
if isinstance(x, bool):
y = x
elif isinstance(x, int):
y = (True, x)
else:
y = x
return y
```
mypy complains about the assignment `y = (True, x)` "error: Incompatible types in assignment (expression has type "Tuple[bool, int]", variable has type "bool")" because it deduces the type of `y` from the first assignment as `bool`.
There are various places in the codebase where we do this kind of untagged union dispatch, this could be fixed by explicitly typing all the variables as `Union[a, b, c, ...]` but I am not sure that in the end it would be worth it. If we install `numpy` and the `pandas-stubs` package (which provides type stubs for pandas) then we get about 120 errors of this nature.
I think that fixing these things is rather difficult, the right approach is to use tagged unions, but there's no support for that in python and any workaround would (I think) make the code unnecessarily non-idiomatic.
If we think it's worthwhile pursuing this, I can prepare a draft patch for some of the uncovered typing issues so we can discuss more concretely.
Contributor guide
Assessment
This issue has not been assessed yet.