microsoft / microsoft/amplifier

Race condition in ~/.amplifier/cache population — concurrent processes clobber in-progress git clones (no cross-process locking)

Open
#325 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.1k
Forks
261
Avg merge
3h 28m
Merged PRs (30d)
13

Description

Symptom (during amplifier update)

✗ Bundle: attractor: Failed to clone https://github.com/microsoft/amplifier-bundle-attractor@main:
Cloning into '/home/dan/.amplifier/cache/amplifier-bundle-attractor-10534381a6383d20'...
fatal: could not open '/home/dan/.amplifier/cache/amplifier-bundle-attractor-10534381a6383d20/.git/objects/pack/tmp_pack_pfD3Ow' for reading: No such file or directory
fatal: fetch-pack: invalid index-pack output

Root Cause Analysis (Verified Evidence)

The tmp_pack_* file vanishing mid-clone is the signature of the target directory being deleted/recreated underneath an in-progress git clone by another process. Not disk-full (42G free), not network.

Timeline of concurrent activity at time of failure:

  • At 19:17–19:18, five interactive amplifier resume sessions were running (started at 19:14)
  • At 19:18+, an evaluation runner spawning amplifier run subagents with --concurrency 8
  • Each process can lazily populate/refresh the cache at startup
  • Your amplifier update was simultaneously cloning into amplifier-bundle-attractor-10534381a6383d20

Cache state after failure (showing the race winner):

  • Cache directory exists and is fully healthy: git fsck clean
  • HEAD = 74a743ad5668c7bf806cea6b88d23ae5a536c97b — exactly matches remote main
  • .amplifier_cache_meta.json records cached_at: 2026-07-15T19:20:41 — written by a different process whose clone succeeded seconds after your clone was killed
  • Two concurrent actors raced remove-then-clone on the identical cache path; the loser (the update) saw its temp pack file deleted mid-operation

Evidence artifact: The pack file list confirms a complete, indexed pack was written:

-r--r--r-- pack-dec2f91b890e158453d66f5c70dd14142fb5b5f8.idx
-r--r--r-- pack-dec2f91b890e158453d66f5c70dd14142fb5b5f8.pack
-r--r--r-- pack-dec2f91b890e158453d66f5c70dd14142fb5b5f8.rev

Impact

  • amplifier update reports "completed with errors" and alarms users, even though the cache ends up healthy (the race winner fetched the same commit as the update would have)
  • Latent risk: The race could also leave a corrupt or partial cache if timing differs, breaking editable installs that point into the cache

Suggested Fix

  1. Cross-process file locking around cache population per cache-dir key
  2. OR: Clone into a temp directory followed by atomic rename, so concurrent populators either wait (if locking) or safely no-op (if one successfully renamed already)

This is a classic producer-consumer race on a shared cache directory without synchronization primitives.

Environment

  • Linux WSL2 (kernel 6.6.87.2-microsoft-standard-x86_64)
  • Amplifier installed via uv tool install
  • Filesystem: /dev/sdd 492GB, 92% used (42GB free)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace cache population from amplifier update and the startup paths used by amplifier resume and amplifier run, focusing on operations targeting the shared ~/.amplifier/cache directory. Reproduce concurrent population if possible, then verify that locking or atomic replacement prevents an in-progress clone from being deleted and that amplifier update completes without the reported error.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.