Parallel installs can fail to resolve setuptools due to a SeparateBodyFileCache race

Description

We are hitting intermittent setuptools resolution failures during parallel source-package installs with Poetry 2.4.1 in CI. I also reproduced the underlying empty-index failure with Poetry 2.4.3, the latest release at the time of reporting, and CacheControl 0.14.4.

The failing CI install was preparing argparse 1.2.1 and bravado-core 6.1.1. The package-index request succeeded, but Poetry reported:

Because -root- depends on setuptools (>=40.8.0) which doesn't match any versions, version solving failed.
Cannot resolve build-system.requires for bravado-core.
Package setuptools (84.0.0) not found.

The local reproducer demonstrates a cache race consistent with this symptom: SeparateBodyFileCache publishes metadata before the separate body file exists. A concurrent Poetry repository lookup can interpret the incomplete entry as an empty package index, either on a fresh-cache hit or during HTTP 304 revalidation. The exact interleaving in the original CI job was not captured.

Expected behavior: concurrent lookups should return the available package links, wait for a complete entry, or treat an incomplete entry as a cache miss.

Related upstream work:

• https://github.com/psf/cachecontrol/issues/324 — non-atomic separate-body cache reads/writes.
• https://github.com/psf/cachecontrol/pull/465 — paired read/write locking; its exact commit passes the focused reproduction below.
• https://github.com/python-poetry/poetry/pull/10816 — switched Poetry from FileCache to SeparateBodyFileCache to support large responses.

This report tracks the impact on Poetry and provides reproduction evidence for evaluating the upstream fix or a Poetry-side mitigation.

Reproduction

The standalone script uses a real localhost HTTP server, Poetry's LegacyRepository, the native CacheControl adapter/controller, and the actual HTML parser. It adds synchronization around existing calls to force the writer to pause before body publication. It does not mock package lists or parser results.

Requirements: uv, Git for the optional fixed control, and Python 3.11 (which uv can provision).

curl -fsSL \
  https://raw.githubusercontent.com/igormishsky/cachecontrol/3af1c3c1e03425793d67a7d0a4799b7f4ed01607/scripts/reproduce_poetry_cache_race.py \
  -o reproduce_poetry_cache_race.py

uv run --no-project --python 3.11 \
  --with 'poetry==2.4.3' --with 'cachecontrol[filecache]==0.14.4' \
  python reproduce_poetry_cache_race.py \
  --expect broken --iterations 5 --output baseline.json

--expect broken exits successfully only when the failure is reproduced as expected. Each run uses new temporary cache directories, so it does not depend on a previously corrupt cache, Artifactory, or a restored GitHub Actions cache.

There are four schedules, repeated five times: threads/processes sharing a cache, each with fresh reads and HTTP 200/304 revalidation. The server sends an ETag and Cache-Control: max-age=60, with no Content-Length or Vary header. One reader sees zero links although the response contains setuptools-84.0.0-py3-none-any.whl.

Results

Poetry	CacheControl	Empty-index failures
2.4.1	Released 0.14.4	20/20
2.4.3	Released 0.14.4	20/20
2.4.1	Original #465 commit 7df32057203eb379f51149da204bddd3d3447bdc	0/20

The original upstream commit was tested without the additional changes in my fork. To run that control:

git clone https://github.com/trail-of-forks/cachecontrol.git cachecontrol-465
git -C cachecontrol-465 checkout 7df32057203eb379f51149da204bddd3d3447bdc

uv run --no-project --python 3.11 \
  --with 'poetry==2.4.1' --with 'cachecontrol[filecache]==0.14.4' \
  python reproduce_poetry_cache_race.py \
  --cache-source ./cachecontrol-465 \
  --expect fixed --iterations 5 --output fixed.json

Poetry remains unmodified and caching stays enabled. With that commit the reader waits for body publication, and both workers find the expected link. Subsequent warm-cache lookups make zero additional HTTP requests. The script prints loaded source paths because the checkout still reports the distribution version as 0.14.4.

These are deterministic HTTP/cache/index-parsing tests, not complete application installs or measurements of natural failure frequency. No wheel is downloaded or executed. Separate fault-injection tests still expose interrupted-write and concurrent 304-update gaps in the original #465 commit; the passing result above is limited to the reproduced read-during-publication failure.

Workarounds

poetry --no-cache install ... bypasses the affected HTTP cache. A separate focused test of Poetry 2.4.1's CLI cache-bypass path passed 5/5 overlapping repository lookups. This has not been validated as a full application-build fix in this report. Clearing the cache alone does not prevent recurrence: every failing reproduction starts with a fresh temporary cache.

Poetry Installation Method

Other: uv-managed isolated environments with pinned Poetry and CacheControl versions.

Operating System

Local reproduction: macOS 15.7.7, arm64, Python 3.11.5.
Observed CI symptoms: Amazon Linux 2023 on arm64 and amd64, Poetry 2.4.1.

Poetry Version

Poetry (version 2.4.3)

Also tested with Poetry 2.4.1, as shown above.

Poetry Configuration

The standalone reproducer constructs Config(use_environment=False) and overrides only cache-dir with a fresh temporary directory and keyring.enabled with false. It uses a localhost LegacyRepository, with no credentials or plugins. The two concurrent workers are explicitly created by the harness.

Relevant defaults, confirmed using an isolated poetry config --list:

installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = false
requests.max-retries = 0
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = null

Example pyproject.toml

Not required for the standalone reproduction: it exercises Poetry's repository lookup directly. The original symptom occurs during isolated build-system dependency resolution for source distributions.

Poetry Runtime Logs

The CI error excerpt is above. The standalone script directly exercises the repository API rather than invoking poetry install, and records actual HTTP requests and parsed links in its output JSON.

Selected summary fields from the Poetry 2.4.3 baseline run:

{
  "poetry_version": "2.4.3",
  "cachecontrol_distribution_version": "0.14.4",
  "expectation": "broken",
  "total": 20,
  "empty_indexes": 20,
  "both_workers_succeeded": 0
}

The HTTP sequence is [200] for a fresh read and [200, 304] for revalidation. In each baseline trial the writer finds the wheel link, while the reader returns [] before the writer is released. With original #465, both return the wheel link and the reader completes after the writer is released.

Start with the linked scripts/reproduce_poetry_cache_race.py and run its Poetry 2.4.3/CacheControl 0.14.4 baseline, then compare the original #465 control. Trace the Poetry repository lookup and cache configuration involved; done means deciding, with a focused regression test if appropriate, whether the upstream fix fully resolves the empty-index result or a Poetry-side mitigation is required.