TST: rare flakyness (or pollution) in a recently introduced subprocess-based test
- Dominant language
- Python
- Stars
- 5.3k
- Forks
- 2.2k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 81
Description
Looking for other sources of test pollution locally, I hit (I assume by chance) a failure mode for this test, introduced in #17598
https://github.com/astropy/astropy/blob/768c5ff1383b9498d2f973663a54305248527b8b/astropy/tests/tests/test_imports.py#L46-L56
It is *extremely* puzzling to me that this test might fail in a subprocess, let alone have anything to do with pollution, but unfortunately it is also very hard to debug; I don't even have a good traceback to work with. All I have is
```
subprocess.CalledProcessError: Command '['/Users/clm/dev/astropy-project/coordinated/astropy/.venv/bin/python3', '-c', "\nimport astropy, sys\nassert 'astropy.units' not in sys.modules\nastropy.units\nassert 'astropy.units' in sys.modules\n"]' returned non-zero exit status 1.
```
I don't have a clear idea how to rewrite this test to at least make it easier to debug when it fails. I tried using `monkeypatch.delitem(sys.modules, "astropy.units", raising=False)`, but it doesn't suffice to simulate "astropy.units has not been imported yet" and the second assert statement fails.
_Originally posted by @neutrinoceros in https://github.com/astropy/astropy/pull/17598#discussion_r1953236865_
ping @mhvk
Contributor guide
Research direction
Start with astropy/tests/tests/test_imports.py, especially the subprocess-based test around lines 46–56, and reproduce the reported CalledProcessError locally. Investigate why the subprocess import-state assertions can fail and whether test pollution is involved. Done means the test reliably checks the intended import behavior and provides enough failure information to debug future failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100