Env-check cache tests share one SQLite file in $HOME across xdist workers
- Dominant language
- Python
- Stars
- 60
- Forks
- 18
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 38
Description
**From Claude** (Anthropic's Claude Code), prompted by @petebachant while reviewing #1579.
While running the suite during #1579 I hit two failures I could not
reproduce, both in julia-manifest cache tests, both passing in isolation
and in a targeted stress run:
- `test_cache_tracks_julia_manifest`
- `test_cache_prefers_versioned_manifest`
Each failed once in a full `pytest calkit/tests` run and passed in three
subsequent full runs, five targeted parallel runs, and on the untouched
branch tip. Neither is touched by that PR.
Separately, `test_conda.py::test_check_env` and
`test_conda.py::test_check_env_editable` each failed once and passed in
isolation. The conda ones are slow (~60s) and depend on a working local
conda, so they may just be environment-sensitive here rather than flaky in
CI.
## Suspected cause for the julia ones
`get_cache_db()` (`calkit/environments.py`) opens a SqliteDict at
`~/.calkit/env-checks/cache.sqlite` -- a single file in the developer's
real home directory, shared by every xdist worker process:
```python
def get_cache_db(name="cache") -> SqliteDict:
env_check_cache_dir = os.path.join(
os.path.expanduser("~"), ".calkit", "env-checks"
)
```
Keys are `f"{wdir}::{env_name}"` and each test gets its own `tmp_dir`, so
this isn't a key collision -- but concurrent multi-process writes to one
SQLite file are a plausible source of intermittent failure, and it also
means the suite reads and writes state belonging to whoever runs it.
## Suggested fix
Point the cache at a tmp path during tests (monkeypatched `HOME`, or a
`CALKIT_ENV_CHECK_CACHE_DIR` env var honoured by `get_cache_db`). That
removes the cross-worker sharing and stops the test suite touching the
developer's real `~/.calkit`.
Flagging rather than fixing since I could not reproduce it reliably enough
to confirm the diagnosis.
Contributor guide
Research direction
Start in calkit/environments.py at get_cache_db(), then inspect test_cache_tracks_julia_manifest, test_cache_prefers_versioned_manifest, and the two test_check_env tests. Run the named tests and a parallel pytest calkit/tests run to observe cache behavior. Done means tests use an isolated temporary cache rather than the developer’s real ~/.calkit cache and no longer share state across workers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- databases, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100