`dmypy` - poor performance when a lot of possibly unrelated errors

Bug Report

The dmypy daemon will experience poor performance if there are a lot of files with errors, even if only a single file is updated.

To Reproduce

Create a ball of mud with errors. This script requires some tweaking to really bring out the issue but its a good starting point:

https://github.com/JamesHutchison/mypy/blob/dmypy-update-perf/ball_of_mud/generate_ball_of_mud.py

The performance deficit is proportionate to the number of failing files.

Expected Behavior

dmypy is always fast for small changes

Actual Behavior

dmypy is slow.

Your Environment

• Mypy version used: 0.991

Why is this important?

When converting a large repo to mypy you will have a lot of failing files at first and you cannot use --follow-imports=silent

Notes

The cause appears to be this code:

https://github.com/python/mypy/blob/master/mypy/server/update.py#L847

From what I can tell, it looks like dmypy relies on files failing their imports to know to revisit them later. This appears to also be the reason that you cannot use --follow-imports=silent with dmypy.

For example, when --follow-imports=error:

• File A is built, and File A imports file B. File A gets errors about B not existing.
• File B is built
• File A gets built again, but this time B is cached (or loaded, or whatever) and no longer fails import

mypy/server/update.py の 847 行目付近から始め、issue にリンクされている ball_of_mud/generate_ball_of_mud.py スクリプトを使って低速化を再現します。1 ファイルだけが変更された場合に、多数の失敗したファイルが再度処理される理由を調査します。無関係なエラーがあっても小さな変更に対して dmypy が高速なままであり、関連する import の動作が維持されていれば完了です。