facebook / facebook/pyrefly

Generic `@overloads` not matched against return type of `Callable` passed as parameter

Open
#3,313 3 comments 0 reactions 1 assignee Claimed by @samwgoldman View on GitHub
overloads stale
Dominant language
Rust
Stars
7k
Forks
516
PR merge metrics
No merged PRs in 30d

Description

### Describe the Bug

I'm not sure if this is similar to #3257, but it is still reproducible in the sandbox. Repro:

```py
from collections.abc import Callable
from typing import Any, Generic, NamedTuple, TypeVar, overload, reveal_type

_T = TypeVar("_T")
_T1 = TypeVar("_T1")
_T2 = TypeVar("_T2")
_T3 = TypeVar("_T3")
_ResultT = TypeVar("_ResultT")

class LResult(NamedTuple, Generic[_T]):
li: list[_T]
v: _T

class TResult(NamedTuple, Generic[_T]):
tu: tuple[_T, ...]
v: _T

@overload
def expand(
callback: Callable[[LResult[_T]], list[_ResultT] | _ResultT],
source: list[_T],
**state: Any, # noqa: ANN401
) -> list[_ResultT]: ...
@overload
def expand(
callback: Callable[[TResult[_T]], list[_ResultT] | _ResultT],
source: tuple[_T, ...],
**state: Any, # noqa: ANN401
) -> list[_ResultT]: ...
@overload
def expand(
callback: Callable[[LResult[_T1], TResult[_T2]], list[_ResultT] | _ResultT],
source1: list[_T1],
source2: tuple[_T2, ...],
**state: Any, # noqa: ANN401
) -> list[_ResultT]: ...
@overload
def expand(
callback: Callable[[TResult[_T1], LResult[_T2]], list[_ResultT] | _ResultT],
source1: tuple[_T1, ...],
source2: list[_T2],
**state: Any, # noqa: ANN401
) -> list[_ResultT]: ...
# @overload
# def expand(
# callback: Callable[..., Any],
# *sources: list[Any] | tuple[Any, ...],
# **state: Any, # noqa: ANN401
# ) -> list[Any]: ...
def expand(
callback: Callable[..., Any],
*sources: list[Any] | tuple[Any, ...],
**state: Any,
) -> list[Any]:
return []

l_i = list(range(10))
t_i = tuple(range(10))
l_f = list(i + 0.0 for i in range(10))
t_f = tuple(i + 0.0 for i in range(10))

def callback_one_l_i(one_l_i: LResult[int]) -> list[int] | int:
return one_l_i.li if one_l_i.li else one_l_i.v

reveal_type(expand(callback_one_l_i, l_i)) # should be list[int]

def callback_one_l(one_l: LResult[_T]) -> list[_T] | _T:
return one_l.li if one_l.li else one_l.v

reveal_type(expand(callback_one_l, l_i)) # should be list[int]
reveal_type(expand(callback_one_l, l_f)) # should be list[float]

def callback_one_t_i(one_t_i: TResult[int]) -> list[int] | int:
return list(one_t_i.tu) if one_t_i.tu else one_t_i.v

reveal_type(expand(callback_one_t_i, t_i)) # should be list[int]

def callback_one_t(one_t: TResult[_T]) -> list[_T] | _T:
return list(one_t.tu) if one_t.tu else one_t.v

reveal_type(expand(callback_one_t, t_i)) # should be list[int]
reveal_type(expand(callback_one_t, t_f)) # should be list[float]

def callback_two_l_t(one_l: LResult[_T], two_t: TResult[_T]) -> list[_T] | _T:
return [*one_l.li, *two_t.tu] if one_l.li or two_t.tu else two_t.v

reveal_type(expand(callback_two_l_t, l_i, t_i)) # should be list[int]
reveal_type(expand(callback_two_l_t, l_f, t_f)) # should be list[float]

def callback_two_t_l(one_t: TResult[_T], two_l: LResult[_T]) -> list[_T] | _T:
return [*one_t.tu, *two_l.li] if one_t.tu or two_l.li else two_l.v

reveal_type(expand(callback_two_t_l, t_i, l_i)) # should be list[int]
reveal_type(expand(callback_two_t_l, t_f, l_f)) # should be list[float]
```

Result:

```
ERROR sandbox.py:70:19-40: No matching overload found for function `expand` called with arguments: ([_T](one_l: LResult[_T]) -> list[_T] | _T, list[int]) [[no-matching-overload](https://pyrefly.org/en/docs/error-kinds/#no-matching-overload)]
Possible overloads:
(callback: (LResult[_T]) -> list[_ResultT] | _ResultT, source: list[_T], **state: Any) -> list[_ResultT] [closest match]
(callback: (TResult[_T]) -> list[_ResultT] | _ResultT, source: tuple[_T, ...], **state: Any) -> list[_ResultT]
(callback: (LResult[_T1], TResult[_T2]) -> list[_ResultT] | _ResultT, source1: list[_T1], source2: tuple[_T2, ...], **state: Any) -> list[_ResultT]
(callback: (TResult[_T1], LResult[_T2]) -> list[_ResultT] | _ResultT, source1: tuple[_T1, ...], source2: list[_T2], **state: Any) -> list[_ResultT]
ERROR sandbox.py:71:19-40: No matching overload found for function `expand` called with arguments: ([_T](one_l: LResult[_T]) -> list[_T] | _T, list[float]) [[no-matching-overload](https://pyrefly.org/en/docs/error-kinds/#no-matching-overload)]
Possible overloads:
(callback: (LResult[_T]) -> list[_ResultT] | _ResultT, source: list[_T], **state: Any) -> list[_ResultT] [closest match]
(callback: (TResult[_T]) -> list[_ResultT] | _ResultT, source: tuple[_T, ...], **state: Any) -> list[_ResultT]
(callback: (LResult[_T1], TResult[_T2]) -> list[_ResultT] | _ResultT, source1: list[_T1], source2: tuple[_T2, ...], **state: Any) -> list[_ResultT]
(callback: (TResult[_T1], LResult[_T2]) -> list[_ResultT] | _ResultT, source1: tuple[_T1, ...], source2: list[_T2], **state: Any) -> list[_ResultT]
ERROR sandbox.py:81:19-40: No matching overload found for function `expand` called with arguments: ([_T](one_t: TResult[_T]) -> list[_T] | _T, tuple[int, ...]) [[no-matching-overload](https://pyrefly.org/en/docs/error-kinds/#no-matching-overload)]
Possible overloads:
(callback: (LResult[_T]) -> list[_ResultT] | _ResultT, source: list[_T], **state: Any) -> list[_ResultT] [closest match]
(callback: (TResult[_T]) -> list[_ResultT] | _ResultT, source: tuple[_T, ...], **state: Any) -> list[_ResultT]
(callback: (LResult[_T1], TResult[_T2]) -> list[_ResultT] | _ResultT, source1: list[_T1], source2: tuple[_T2, ...], **state: Any) -> list[_ResultT]
(callback: (TResult[_T1], LResult[_T2]) -> list[_ResultT] | _ResultT, source1: tuple[_T1, ...], source2: list[_T2], **state: Any) -> list[_ResultT]
ERROR sandbox.py:82:19-40: No matching overload found for function `expand` called with arguments: ([_T](one_t: TResult[_T]) -> list[_T] | _T, tuple[float, ...]) [[no-matching-overload](https://pyrefly.org/en/docs/error-kinds/#no-matching-overload)]
Possible overloads:
(callback: (LResult[_T]) -> list[_ResultT] | _ResultT, source: list[_T], **state: Any) -> list[_ResultT] [closest match]
(callback: (TResult[_T]) -> list[_ResultT] | _ResultT, source: tuple[_T, ...], **state: Any) -> list[_ResultT]
(callback: (LResult[_T1], TResult[_T2]) -> list[_ResultT] | _ResultT, source1: list[_T1], source2: tuple[_T2, ...], **state: Any) -> list[_ResultT]
(callback: (TResult[_T1], LResult[_T2]) -> list[_ResultT] | _ResultT, source1: tuple[_T1, ...], source2: list[_T2], **state: Any) -> list[_ResultT]
ERROR sandbox.py:87:19-47: No matching overload found for function `expand` called with arguments: ([_T](one_l: LResult[_T], two_t: TResult[_T]) -> list[_T] | _T, list[int], tuple[int, ...]) [[no-matching-overload](https://pyrefly.org/en/docs/error-kinds/#no-matching-overload)]
Possible overloads:
(callback: (LResult[_T]) -> list[_ResultT] | _ResultT, source: list[_T], **state: Any) -> list[_ResultT]
(callback: (TResult[_T]) -> list[_ResultT] | _ResultT, source: tuple[_T, ...], **state: Any) -> list[_ResultT]
(callback: (LResult[_T1], TResult[_T2]) -> list[_ResultT] | _ResultT, source1: list[_T1], source2: tuple[_T2, ...], **state: Any) -> list[_ResultT]
(callback: (TResult[_T1], LResult[_T2]) -> list[_ResultT] | _ResultT, source1: tuple[_T1, ...], source2: list[_T2], **state: Any) -> list[_ResultT] [closest match]
ERROR sandbox.py:88:19-47: No matching overload found for function `expand` called with arguments: ([_T](one_l: LResult[_T], two_t: TResult[_T]) -> list[_T] | _T, list[float], tuple[float, ...]) [[no-matching-overload](https://pyrefly.org/en/docs/error-kinds/#no-matching-overload)]
Possible overloads:
(callback: (LResult[_T]) -> list[_ResultT] | _ResultT, source: list[_T], **state: Any) -> list[_ResultT]
(callback: (TResult[_T]) -> list[_ResultT] | _ResultT, source: tuple[_T, ...], **state: Any) -> list[_ResultT]
(callback: (LResult[_T1], TResult[_T2]) -> list[_ResultT] | _ResultT, source1: list[_T1], source2: tuple[_T2, ...], **state: Any) -> list[_ResultT]
(callback: (TResult[_T1], LResult[_T2]) -> list[_ResultT] | _ResultT, source1: tuple[_T1, ...], source2: list[_T2], **state: Any) -> list[_ResultT] [closest match]
ERROR sandbox.py:93:19-47: No matching overload found for function `expand` called with arguments: ([_T](one_t: TResult[_T], two_l: LResult[_T]) -> list[_T] | _T, tuple[int, ...], list[int]) [[no-matching-overload](https://pyrefly.org/en/docs/error-kinds/#no-matching-overload)]
Possible overloads:
(callback: (LResult[_T]) -> list[_ResultT] | _ResultT, source: list[_T], **state: Any) -> list[_ResultT]
(callback: (TResult[_T]) -> list[_ResultT] | _ResultT, source: tuple[_T, ...], **state: Any) -> list[_ResultT]
(callback: (LResult[_T1], TResult[_T2]) -> list[_ResultT] | _ResultT, source1: list[_T1], source2: tuple[_T2, ...], **state: Any) -> list[_ResultT] [closest match]
(callback: (TResult[_T1], LResult[_T2]) -> list[_ResultT] | _ResultT, source1: tuple[_T1, ...], source2: list[_T2], **state: Any) -> list[_ResultT]
ERROR sandbox.py:94:19-47: No matching overload found for function `expand` called with arguments: ([_T](one_t: TResult[_T], two_l: LResult[_T]) -> list[_T] | _T, tuple[float, ...], list[float]) [[no-matching-overload](https://pyrefly.org/en/docs/error-kinds/#no-matching-overload)]
Possible overloads:
(callback: (LResult[_T]) -> list[_ResultT] | _ResultT, source: list[_T], **state: Any) -> list[_ResultT]
(callback: (TResult[_T]) -> list[_ResultT] | _ResultT, source: tuple[_T, ...], **state: Any) -> list[_ResultT]
(callback: (LResult[_T1], TResult[_T2]) -> list[_ResultT] | _ResultT, source1: list[_T1], source2: tuple[_T2, ...], **state: Any) -> list[_ResultT] [closest match]
(callback: (TResult[_T1], LResult[_T2]) -> list[_ResultT] | _ResultT, source1: tuple[_T1, ...], source2: list[_T2], **state: Any) -> list[_ResultT]
INFO sandbox.py:65:12-43: revealed type: list[int] [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:70:12-41: revealed type: Unknown [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:71:12-41: revealed type: Unknown [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:76:12-43: revealed type: list[int] [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:81:12-41: revealed type: Unknown [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:82:12-41: revealed type: Unknown [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:87:12-48: revealed type: Unknown [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:88:12-48: revealed type: Unknown [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:93:12-48: revealed type: Unknown [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
INFO sandbox.py:94:12-48: revealed type: Unknown [[reveal-type](https://pyrefly.org/en/docs/error-kinds/#reveal-type)]
```

Un-commenting the commented `@overload` will eliminate the errors, but will match most call sites (revealed type will go from `Unknown` to `list[Any]`, which still isn't right.

Pyright gets this right: [pyright playground](https://pyright-play.net/?code=GYJw9gtgBAxmA28CmMAuBLMA7AzgOgEMAjGKdCABzBFSgGEDFjkAoUSKVATwvSwHMylarQCCWLgBooAcSRYkIdDGkA5AhCQATACoBXCsmk6eSAGoEQ0sADdF8MAS3SQSO4wD63CkhYsPOlAAvFAmPhYgABQARAHRAJT%2BOgCMwaGmETEByQlJAExpYeaWWTp5uQEAzIUZJbE6lRUASkg4evCogSFFmbEtbR06uSww8AQ4OFAAMv3tqJHqmroGRrLyisoA2gEAuvEAXCxQx1Dw6Pun6Dio2zo7Ryc2FwF%2Bo%2BOTOrMdCxra%2BoZIaRyBRKGC3PaHE6cPQXVArJC3aR4ZH3KFPKAvFgAAVs9kcWhYWiQwCgSAAHhQCFgtJEHscYIx4EQCDAANYXBhMIjITabGatObgnbSM7XbZfTo7KAAHwxErukjpUBwYD0IBgSAuopuu0VUIAVPrrgRUJqoOIpMcAMRQLBgACOBAuolUqgALAAGZIseJQAC0AD5LmKPPKdhdkXhsbiQA4nITiaSKVSaUqGYhmWyOYzmAjNp8BR0hSKrjqwzK5YXJXqTiq1RrYfDEVBI8KlYbjabnRJpFAbXbHc7XZ7vb7A8Gy1W7hHkdG7LH8QmSeTKdTaVD00yWez6DnuXn%2BQMdSlhaEJbc8jtT9rxVOpbLQ3ea8c6%2BqkMktaXbsk21DXxq8kbAELyRFFnygDtUBNM0LV7fsHSdc1hy9H1-SDG9HyPacW1nHF5zjAkiWXZM1zTRlMx3Tkxn3XkCyPb9T0PQUAkva8v0wuY7grDjBl-WtVTfD9oWA7JQLwPiXwEgDPxDMoJIgo0oK7c0e2tW0EKHd0ULHdD2LDGcoyIpNV1TDdyO3bMuR5SNpAteSjSk1oZJuOyKzhYDYJw8TwMg6DuykVDxxvOzIROVw4RALAoE2e4WHgDx0DSbVIhAKl%2BCQSJkg9eJElQBK0nc5AUrSjKspyuKPBJEJksSgBqKAPTwD0oGAagyDIKLUoEUrstyyqCvhSI6oapqWraxK%2BCgLr0sy3q-CMzcKI8bAkA8eL0EiFa1oSi4mKLPhUD2NCJ02A77w61BQuOcK1Sirb1rwM4yBJe6EsexKkHgHAkCgV70DwGw-Fcdx4u8DKVxTSJFu3ZaFG29ARQSnK1JwAALVV4C0KAiB%2Bm8zvmxNobZWHVvgTa4fgXbz12HSTt2bidCuqakAiu6Kfe57fvZp7Pu%2BrnSYBoG3CQTwwciCG1yJ1kSbWxH0GRvtlXR9osZxk78eBkXQdMcWSJpKWZfgRHgAVm00Yx1Xca-YA40OgmSQNra8o2p2drPKtTqwQ7abxr3zoOpmbsiidydW528DhX10BeuHw7hUkvp%2B13-sBlhNdFnWJf18zieT6RndNpWLexq2xXxpdYBz6WndDrwLjo5i7h99iuIfRmlSDqLkqdiO9CjmOw97hO%2BZ71P0%2B1nxdZMqGq5l1B86R31FfNlWS-Vv20%2BFjPJ6zmeMxhp2F5NpezeVzG15vG3HDtiuDdQAB3MBtvme6qY93VOEfuv3fomnjow1uGJ25Qk7tFfU913rSH1A-J%2BqBe5SmjvzNaHM2owK8EPXmP00FwLHlvCe4M9Z7y3MTNBoM5YL3lifIuq81a%2BztuPLwmdCF3y-mQ04lUj6FxXufWh1tbaxQWrPbBa1a6XR-o3U8pC36-ybv-Fu50AiBxZrdMBPc4RQNIe9BBA90Hx1QawjmmDP5P3gILTeINGE72YUIr%2BeUjacASnLLhZ9Lbr3oXgyxBDp4sNgbLBxwBjbOOLrwsUV8TT3CAA)

### Sandbox Link

https://pyrefly.org/sandbox/?project=N4IgZglgNgpgziAXKOBDAdgEwEYHsAeAdAA4CeS4ATrgLYAEAxrlLAwC4S7pyGrYN0INYrkps6AYVQs%2BsADrow1em1LEI6AOaDho8QEF0pADR0A4jHQxKEBqYByqGjEwAVAK7FYp12pgA1VEpTXAA3ayhcVExTShhw6QB9VWIYBQVE1zoAXjpfVMDKAAo5EEzSgEoM1wBGHLy-QpKy2srqgCZ6-ICg5sz2tvRMgGYuxt7SkcHEgCV4dyg2LNzupsm5uAWlwYUGKFQ4ODoAGQ2tosdnN09vc0trWwBtTIBdCsQFOi%2B6KAhEH4gcDYz1cL0%2B31C-3K6F2%2B0OeTOiwuThcHi8MFMFisNgYILeH3Q3zobHc-xJ6JBpkI1LBhIhUNc6XQAAEwhEopgFJgYGA6DB8MQMJgSnSvgxpFBsKgGABrf5SGTYWCPR6neaLPEvUy-IHPRFLF50AA%2BdFm6oNxnBXzguHclAYMH%2BOuBr0toroACoPUDUGxHXRDCYvgBiOjoXAAR1Q-309nsABYAAw1BQVOgAWgAfADdWbNotQf9qYQFKzwpRItEuTy%2BQKhSKieKWFLZfKJbIYCrXPrNdrAS79aDjabB1qrXQbXaHWSbp3MlSaW6iV6fX6Y0ZTHRQ%2BGozG40mU%2Bg01mcwPzYW6MXS2yKxzq7z%2BYKsA3vk3JdK5ZJ20rO6qe5kai1BFzRBdoXiA509XPQ0TTzLZQSXb5J3tGAaidfsQUAxDrVtFD2hnCl%2BgXQgx3dFc2F9f1A03bdI2jAN92TVMM2zSC4ILF4i2pa9y0rTksBrR963HN8W0-BV9h-Lt-1qIC1XzF1XDAiCMPYg1hzUhDx2Qh00OJWdMOI0iiR0mB8NPUDjO%2BcjKPXIMtzDOi9wTJijxYizNM4y9uPQUMy3ZKtfLobkHzrZ8FFDRsJTEttFWVYtTEDUjIus0y4HQ3UkuHcllWo7ySLdFKvhstcAw3ENHN3BiXMPUNj1YjCkq4ksBNCp9hRE6KP1iyT4upRKjCs4q0oy4EspNHLOzy4shs9b0KNK6jmJPSCmvHOISUoQlHlpBQoESCB6mdIpKAwTQYCKGpEwqKp0DYA76kmk6zouq6br2xJeVyY7DoAajoRNCETOgwFEQRBEJU6tFe67bvur79PRIo-oBoGQbBw6NDoKHzsu2GmRCxgutlRIuBgRJ9ogIoyYpg7-nkrZHg0Ng3ncyDmZgiG2AJIkNrtQkacpwhfkEXlBYO4XDpgKA4BgOhxYgQhQiZOIEn2lILqE59RI-UmrFpiBtQOm6KrgAALW0oEwOhsDl9m7t21qiebXXBep-WoHpmTWZW1Sh1g1wee%2BPmtvlj3JdFsPyagCPpdlqOKaVlX4hgJINaKLXhR1knBaNiATYc83Let22LI5hRVdT9W-AzsKs%2BJmU9ejo2wAL0Mi4WEu7YwsBKxZgma2zxuafuqmR7p4CFKZh36rLh3h2ZoOvhDwljvHxWSTTCAxf10fCBJPkZbl9ek5hdBK7TmvM6KIem%2BSA7TFHtuJwtzube73Vy7Pwnb5H93ye5pPRmrxZ5sX9qaQO60YCbVXv2f%2ByR97uC3jvABiDD7xxHqfCuKdL6pFru1G%2BDc75sEfsbNMhdX5W3fnPfu58cHVzwdfX%2Bu9SGt3Ie3ShXcLK9yiLQ%2B8zt3wkzYAAd1wLTNg8DPYnG9o-URyR-jdhAiAtmftOaZCXtjaB-M6CPA9ILSWpgPQiLEWwRBhpt4JxjiLMGxiEEHzjnLWxpjlZnwvgwzWddCEuyEXI9WedSH53YS-Yu1D7a0LcckK%2Bnjb62L8T8T6rDn4dyoaXSCPDfSOx-kQpxFN4GAMUVPV0xJfFeyUaCUBqiNKQPdCvHRejd6IMMbEyW5iUF2PcPLSgxSxFWKlkfbpicXHYLVpExh0TslyPulAAJeckmcNCRhL%2BET05MImSYimrCW5zJCaknufdaQgGMCAMgcRe6kH3rQKAFBQwAAVSCnKgKQCcQo8D4EYFwSAmg7S%2Bk4OgFqoYADKMA5ZmzYGwYg6UAD0kKTk8keYQUQmhIWWEhZgXADA4CQqYIoCAXzTocC4JC9GXTUChFQNADs7ycV4p%2BVweWxACXcBamQNgFt0DpnLHAX59RSjDEIDUAYIAFCPGsNQSgcBaTuHQEIEQYgXDpkwBAOI7AIDhHqAAclxeGOI6qFDhjYOmOIEZ3BKvlTKGApB0zSgdPCXI6rhFBHQLq9AIAAC%2BRzpQcHCAAMWgDACgaAsCvJIOQV1QA

### (Only applicable for extension issues) IDE Information

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.