Definition of `str.split`, `str.rsplit` and `str.splitlines` results in false positive errors
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.1k
- Forks
- 2.1k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 82
Description
I recently fixed a bug in pyright that caused a false negative when attempting to assign a list[LiteralString] to a list[str]. As we know, the type parameter for list is invariant, so these types are not consistent.
from typing import LiteralString
v1: list[LiteralString] = ["a", "b"]
v2: list[str] = v1 # Should be a type violation
With this bug fix now in place, the definitions for the methods str.split, str.rsplit and str.splitlines are problematic. These three methods are defined using overloaded signatures, and the first overload returns list[LiteralString]. This means the following code now results in a type violation.
def process(x: list[str]) -> str:
return " ".join(x)
process("1,2,3".split(",")) # Type violation
This is going to generate a lot of new type errors in existing code bases.
I initially attempted to work around this by adding a heuristic in pyright's overload matching logic so it deprioritized the first overload in this case, but this heuristic broke other code and really isn't sound. I've therefore decided to remove the heuristic.
I don't have any good ideas about how to fix this, but I wanted to open the issue to start the discussion to see if anyone had a suggestion. If we're going to fix it, I think the fix needs to be in the typeshed stubs. I don't see a way to fix this in a type checker — at least while remaining true to the type system.
One possible solution is to omit the LiteralString overloads for these three methods. This would eliminate the type violation error in the common case, but it would introduce new errors in cases where callers expect to see a return result of list[LiteralString].
Note that this doesn't currently affect mypy users because mypy has not implemented support for LiteralString.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the overloaded typeshed definitions for str.split, str.rsplit, and str.splitlines, focusing on their LiteralString return types. Use the examples in the issue to reproduce the list invariance errors and assess the alternatives discussed. Done means resolving the false positives without undermining valid LiteralString results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100