python / python/mypy

Avoid type checking dependencies when errors aren't reported

Open
#12,854 2 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature performance
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Feature

When we're type checking code that we aren't going to report errors for (like any code in site-packages), we can improve performance by avoiding doing most of the work of type checking.

Pitch

I tried type checking mypy with an uncompiled mypy from master (19e9f2136fd46c662f1066a4894edf48173942f4) itself using this command:

mypy mypy mypyc --config-file mypy_self_check.ini --no-incremental --timing-stats timing_data.txt

The results I got using the per-file timing stats were that of the approximately 63.6 seconds spent type checking, 44.3 seconds were spent type checking modules in mypy or mypyc, and the remaining 19.3 seconds were spent in other modules.

While 19.3 seconds isn't a majority of the build time, it is still a not insignificant amount of time spent type checking dependencies, even though that work will mostly get thrown away since we don't report errors in any of those dependencies.

There are probably a couple things we can skip doing for code that doesn't have errors reported:

  1. Avoid semantic analysis and type checking inside functions, since that shouldn't affect what other modules see.
  2. Skip particularly expensive checks at module level when type checking (we can't really completely skip type checking because we probably still need to run type inference on that code, and I think we group those together).
  3. Avoid loading modules that are never used directly in the code being checked (currently we sometimes analyze modules that are imported directly or indirectly by another dependency module that is directly used in checked code).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the provided mypy command and timing-stats output, then compare time spent checking dependencies with code whose errors are reported. The work is done when unreported dependency checking is reduced without changing the diagnostics or required type information for directly checked code.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.