RFC: a type comment to ignore expected errors

Problem

Python is optionally typed, and a lot of type-annotated libraries also add runtime type checks to prevent misuse by users who do not type-check their own code. For example:

def launch_missile(name: str) -> None:
    if not isinstance(name, str):
        raise TypeError("name is not an str")
    ....

If the library then wants to test this check, mypy complains:

def test_launch_missile_only_accepts_str() -> None:
    with pytest.raises(TypeError):
        launch_missile(10)  # error: Argument 1 to "launch_missile" has incompatible type "int"; expected "str" [arg-type]

The error is legitimate and expected, but cannot be left alone because it causes mypy to fail. The usual way to fix this is to tell mypy to ignore it:

launch_missile(10)  # type: ignore[arg-type]

This solution for the most part works great, but is not ideal:

One problem with it is that if something in the type of the line changes unintentionally such that the type error no longer triggers, the type: ignore comment doesn't do anything, silently. Hence it is not a good vehicle for doing "type-level assertions".

mypy provides also solution for this unused-ignore problem -- the warn_unused_ignores option. But this in itself is also not ideal: it can be specified either globally or per-file, but that is too coarse. Sometimes there are type: ignores which only conditionally trigger, depending on e.g. python version, dependency version or platform, so warn_unused_ignores cannot be set.

Another problem with using type: ignore for "type-level assertions" is that the intent is not self-evident - it is not immediately clear whether it is meant as an assertion or as a workaround/TODO/I-know-this-is-bad-but-I-am-doing-it-anyway.

When working on adding (inline) type annotations to the pytest project, and also in my personal projects, I have wanted this often.

Solution

Add a new type: ignore-expected directive (also accepts error codes type: ignore-expected[arg-type,operator]).

Semantically, this directive is exactly the same as type: ignore, except that it triggers an error about being unused even if warn_unused_ignores is not set, and has a differentiated error message unused 'type: ignore-expected' comment.

For type-errors that are "asserted"/expected, type: ignore-expected should be used. For other cases, type: ignore should continue to be used.

Practical matters

A POC implementation is available at https://github.com/bluetech/mypy/tree/type-ignore-expected. The implementation is not polished, and is lacking comprehensive tests and documentation, but is meant to show feasibility.

Above I proposed to add a new directive. The existing directive type: ignore is actually parsed and is a part of the Python AST (since Python 3.8, or in typed-ast for earlier versions). It is represented as a TypeIgnore AST node (link). If this idea is accepted, then hopefully the TypeIgnore node can be extended with a expected: bool attribute. In the mean time, due to the loose why the comment is apparently parsed by the parser, it is possible to handle any way in a hacky way, as the PR does. Such an approach can also be used to provide seamless backward compatibility.

Prior work

When researching this I have found out that TypeScript has just added support for exactly this (not in a released version yet at the time of writing), see https://devblogs.microsoft.com/typescript/announcing-typescript-3-9-beta/#ts-expect-error-comments. The equivalent to their directive name would be type: expect-error[...], but is otherwise the same as far as I can tell.

Start by reviewing the linked POC implementation and the Python or typed-ast TypeIgnore representation referenced in the issue. Determine how error codes, unused directives, backward compatibility, comprehensive tests, and documentation should behave; done means the proposed directive is implemented with its differentiated unused-comment error and coverage for the described cases.