python / python/typing

[spec] overload subtyping rules are too strict.

Open
#2,021 3 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: other
Dominant language
Python
Stars
1.8k
Forks
302
Avg merge
23h
Merged PRs (30d)
8

Description

The typing spec states that, when A and B are potentially overloaded methods

If a callable B is overloaded with two or more signatures, it is assignable to callable A if at least one of the overloaded signatures in B is assignable to A

If a callable A is overloaded with two or more signatures, callable B is assignable to A if B is assignable to all of the signatures in A

However, this definition seems to be too strict. Consider the following example:

from typing import Protocol, Self, overload

class IntScalarOurs(Protocol):
    def __add__(self, other: int | Self, /) -> Self: ...

class IntScalarTheirs(Protocol):
    @overload
    def __add__(self, other: int, /) -> Self: ...
    @overload
    def __add__(self, other: Self, /) -> Self: ...

These two protocols specify the exact same runtime behavior. Yet, following the rules of the spec, IntScalarOurs is assignable to IntScalarTheirs, but IntScalarTheirs is not assignable to IntScalarOurs.

Case 1: A=IntScalarOurs, B=IntScalarTheirs.

If a callable B is overloaded with two or more signatures, it is assignable to callable A if at least one of the overloaded signatures in B is assignable to A

  • Is (self, int) -> Self assignable to (self, int | Self) -> Self? No, because int is not a supertype of int | Self (contravariance)
  • Is (self, Self) -> Self assignable to (self, int | Self) -> Self? No, because Self is not a supertype of int | Self (contravariance)

Thus, IntScalarTheirs is not assignable to IntScalarOurs.

Case 2: A=IntScalarTheirs, B=IntScalarOurs.

If a callable A is overloaded with two or more signatures, callable B is assignable to A if B is assignable to all of the signatures in A

  • Is (self, int | Self) -> Self assignable to (self, int) -> Self? Yes.
  • Is (self, int | Self) -> Self assignable to (self, Self) -> Self? Yes.

Thus, IntScalarOurs is assignable to IntScalarTheirs.

From a pure set-theoretic POV, it seems that subtyping functions should follow a very simply rule based on the domain.

$f <: g$ if and only if $\text{dom}(f) ⊇ \text{dom}(g)$ and $f(x) <: g(x)$ for all $x∈\text{dom}(g)$

What is currently specced appears to be a lemma for a sufficient condition, but not a necessary one.
Individual type-checkers can of course use such lemmas if the general case is too difficult to check/implement (and communicate that to their users), but shouldn't the spec be biased towards the theoretically principled point of view, when applicable?

Related Discussions

https://github.com/python/typing/discussions/1782

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the overload subtyping rules in the typing spec's callables section and review the related discussion at typing/discussions/1782. Compare the stated rules with the IntScalarOurs and IntScalarTheirs example, then determine whether the specification needs a principled revision or a clarified sufficient-condition rule.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.