Speed up union simplification
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Union simplification (make_simplified_union) has been causing multiple performance issues (at least #9169, #12408, #12225). It can make proper subtype checks of all union items against all other items, which is O(n**2) -- with certain O(n) fast paths that cover some (but not all) problematic scenarios. Union simplification is fairly performance-critical even when we don't hit worst-case scenarios.
Here are some ideas about what we might do to improve the situation:
- Somehow implement union simplification of multiple
Instancetypes (at least simple ones) in close to linear time. I suspect that this is possible under some reasonable assumptions. - Cache negative results of proper subtype checks. I think that currently we only cache positive results (in
mypy.typestate). This might have some drawbacks, such as a possible explosion of cache sizes. I assume there's a reason why we aren't currently doing this. Union simplification tends to perform many proper subtype checks with negative results. - Avoid doing full union simplification in some cases, perhaps based on some heuristics. Union simplification should never be semantically necessary.
- Add fast paths for the most common union simplification operations (e.g. single item,
X | None).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with make_simplified_union and the subtype-check caching in mypy.typestate, then review the related issues #9169, #12408, and #12225. The work is complete when union simplification is measurably faster without changing its semantics, but the issue does not specify a single implementation or benchmark.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100