Collapsed Optionals in TypeVar cause weird error messages
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
At a minimum, I'm reporting a bug in an error message.
I have a function, run_tasks, which takes a list of tasks and a worker function, and executes the worker function on almost all the tasks. For some of them, run_tasks will decide to not execute the task.
run_tasks returns List[Optional[Result]] where Result is a TypeVar.
This works until the worker function's returns Optional[x] where Result is now Optional[x] and run_tasks is asked to return List[Optional[Optional[x]]].
Example code:
from typing import (
TypeVar,
List,
Callable,
Optional,
)
Task = TypeVar("Task")
Result = TypeVar("Result")
def run_tasks(
tasks: List[Task], worker_fun: Callable[[Task], Result]
) -> List[Optional[Result]]:
results: List[Optional[Result]] = []
toggle: bool = True
todo: Task
for todo in tasks:
toggle = not toggle
if toggle:
work_result: Result
work_result = worker_fun(todo)
results.append(work_result)
else:
results.append(None)
return results
def test() -> None:
def worker(d: int) -> Optional[int]:
if d % 2 == 0:
return None
else:
return d + 1
output: List[Optional[Optional[int]]] = run_tasks(
[1, 2, 2, 1], worker,
)
from pprint import pprint
pprint(output)
assert output == [None, None, None, 2]
test()
I get the error message: error: Argument 2 to "run_tasks" has incompatible type "Callable[[int], Optional[int]]"; expected "Callable[[int], int]"
I understand Python can't really represent Optional[Optional[int]]. The problem is in the error message, I think. This code does work correctly, and validates:
def test2() -> None:
def worker(d: int) -> int:
return d + 1
output: List[Optional[int]] = run_tasks(
[1, 2, 2, 1], worker,
)
from pprint import pprint
pprint(output)
assert output == [None, 3, None, 2]
test2()
and so someone unfamiliar with the intricacies and details may be confused by the type signature being correctly updated in 2 places, but not validating.
Instead, I sort of expected it to either (a) work, or (b) have an error specific to the case of nested Optional.
$ mypy --version
mypy 0.761
$ python --version
Python 3.7.6
flags: none.
My apologies, but I did not test with mypy from Git master.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided run_tasks reproducer with mypy and compare the nested-Optional diagnostic with the working test2 case. Done means the incompatible Callable error is corrected or replaced with a diagnostic that accurately explains this nested Optional case, with regression coverage for the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100