dmyy crashes when switching to git branches which removes subdirectory

Crash Report

I am using dmypy to check code in a pre-commit hook. When switching branches, obviously a lot of files/packages come and go. It seems as if dmypy has problems with the internal cache when a formerly checked file/package is not available anymore.
I have encountered two different backtraces.

Traceback

Daemon crashed!
Traceback (most recent call last):
  File "mypy/dmypy_server.py", line 221, in serve
  File "mypy/dmypy_server.py", line 264, in run_command
  File "mypy/dmypy_server.py", line 323, in cmd_run
  File "mypy/dmypy_server.py", line 385, in check
  File "mypy/dmypy_server.py", line 559, in fine_grained_increment_follow_imports
  File "mypy/dmypy_server.py", line 688, in direct_imports
KeyError: 'removed.package'

Daemon crashed!
Traceback (most recent call last):
  File "mypy/dmypy_server.py", line 221, in serve
  File "mypy/dmypy_server.py", line 264, in run_command
  File "mypy/dmypy_server.py", line 323, in cmd_run
  File "mypy/dmypy_server.py", line 385, in check
  File "mypy/dmypy_server.py", line 577, in fine_grained_increment_follow_imports
  File "mypy/server/update.py", line 1164, in refresh_suppressed_submodules
  File "mypy/fscache.py", line 172, in listdir
  File "mypy/fscache.py", line 168, in listdir
FileNotFoundError: [Errno 2] No such file or directory: '..../deleted/directory'

To Reproduce

Switch between git branches which add/remove a complete subdirectory.

Your Environment

• Mypy version used: 0.812
• dmypy command-line: dmypy run -- --config-file mypy.ini -p package1 -p package2 -p package3
• Python version used: 3.9
• Operating system and version: MacOS

Reproduce el fallo ejecutando el comando dmypy indicado, cambiando entre ramas que añaden o eliminan un subdirectorio completo y comprobando los tracebacks del daemon. Empieza en mypy/dmypy_server.py, en fine_grained_increment_follow_imports, direct_imports y las llamadas a mypy/server/update.py y mypy/fscache.py. Se considera terminado cuando los cambios entre ramas ya no provocan que dmypy falle y el comportamiento de regresión relevante esté cubierto.