race in record_missing_stub_packages (No such file or directory: '.mypy_cache/missing_stubs')
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.6k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
Bug Report
When running many mypy instances in parallel on the same project, our pipeline sometimes fails with the following traceback:
Traceback (most recent call last):
File "/usr/local/bin/mypy", line 8, in <module>
sys.exit(console_entry())
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/mypy/__main__.py", line 15, in console_entry
main()
File "mypy/main.py", line 102, in main
File "mypy/main.py", line 186, in run_build
File "mypy/build.py", line 193, in build
File "mypy/build.py", line 296, in _build
File "mypy/build.py", line 3539, in record_missing_stub_packages
FileNotFoundError: [Errno 2] No such file or directory: '.mypy_cache/missing_stubs'
To Reproduce
Run ten parallel mypy on 10 different files in your project from a cold start. Repeat until it fails.
Expected Behavior
That .mypy_cache/missing_stubs contains garbage or results from a random run. Or that there was a flag to disable missing_stubs writing.
Actual Behavior
See traceback above.
Your Environment
Ubuntu Linux running inside a docker container.
- Mypy version used: mypy 1.12.0, mypy-extensions 1.0.0
- Mypy command-line flags: mypy sourcefile.py
- Mypy configuration options from
mypy.ini(and other config files):
[tool.mypy]
mypy_path = "src"
disable_error_code = "import-untyped"
[[tool.mypy.overrides]]
module = [
"secretpackage.secretmodule1",
"secretpackage.secretmodule2",
]
follow_imports = "skip"
- Python version used: 3.11 (didn't record minor version when it happened)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in mypy/build.py at record_missing_stub_packages, using the traceback and parallel cold-start reproduction as the entry points. Investigate why ten concurrent mypy processes can fail while writing .mypy_cache/missing_stubs. Done means parallel runs no longer raise FileNotFoundError, or the behavior is controllable as described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100