Mypy/dmypy takes over 16GB of memory

Bug Report
Hello!

Happy Friday folks! I am writing as I was hoping someone here might be help with my company's mypy usage. For us, most of the code is in a massive django monolith with hundreds of dependencies and tens of thousands of lines of code. We are trying to do a good job in adding typechecking in as many places in our code base as we can, but our running into a mypy/dmypy limitation. Dmypy is now taking over 10 minutes to run and when fully started is consuming over 16GB of memory. This slowness and memory consumption is causing serious problems for our engineers, even leading to multiple OOMs on their dev machines because of this memory usage. It has gotten so bad that we have started telling folks to stop running mypy.

I do want to see if there is any fix I can use, so I'm writing here to see if anyone in the community knows how to address these kinds of issues. Is there anyway to tell dmypy to use disk, or the max amount of memory it can use? Any speed hacks? I'm open to any idea.

Thank you for reading, and have a nice day!

To Reproduce

1. Have a huge monolith
2. Run dmypy on it

Expected Behavior
Don't take over 50% of the total available memory in a macbook or beefy ec2 🤣

Actual Behavior
Uses over 16GB of memory

Your Environment
Mypy: 1.4.1 (actually got worse in 1.9)
Command: dmypy start --log-file .dmypy.log -- --follow-imports=error --cache-dir=/dev/null --config-file /pyproject.toml --use-fine-grained-cache
mypy.ini:

[mypy]
no_implicit_optional = True
local_partial_types = True
follow_imports = normal
disallow_untyped_defs = True
disable_error_code = misc

Python version: 3.11.8

Inizia dal comando dmypy segnalato, dalle impostazioni di pyproject.toml e dalle opzioni di mypy.ini, quindi confronta il comportamento relativo alla memoria e all’avvio osservato per mypy 1.4.1 e 1.9 su una codebase Django di grandi dimensioni. Riproduci il problema con le impostazioni di cache e import indicate; il lavoro è completato quando viene identificata una causa concreta o una mitigazione per l’uso eccessivo di memoria e il tempo di esecuzione.