Dev Containers: concurrent server installs into the shared `vscode` volume race in `mv -n`, aborting container start and leaving a nested duplicate

• VSCode Version: 1.136.1 (commit a44adf7f53e00964ab890f9f8758a334f1fc15bc)
• Local OS Version: Debian 13, kernel 7.1.8+deb13-amd64
• Remote OS Version: Ubuntu 26.04 (container image)
• Remote Extension/Connection Type: Dev Containers 0.466.0, @devcontainers/cli 0.88.0, rootless podman 5.4.2 (dev.containers.dockerPath = podman)
• Logs: inline below

Summary

When several dev-container windows are restored at once after a VS Code update, they all install the server for the new commit concurrently into the extension's shared vscode named volume (/vscode/vscode-server/bin/linux-x64/<commit>). The install ends with mv -n <tmp>/* <target> && rmdir <tmp>, which is not safe against a concurrent installer:

• the first window renames <tmp>/vscode-server-linux-x64 to <target> - correct;
• the second window finds <target> present, so mv -n moves its extraction into it, creating <target>/vscode-server-linux-x64/, a complete 703 MB duplicate of the server. rmdir succeeds, exit code 0, and the extension reports the install as successful;
• every later window finds <target>/vscode-server-linux-x64 present, so mv -n silently skips (exit 0), rmdir <tmp> then fails with Directory not empty, exit code 1, and the extension aborts the container start.

In the session documented below, 4 of 6 dev containers failed to open and a fifth left a dead nested copy behind.

Steps to Reproduce:

1. Open several workspaces (here: 6) in dev containers. All of them share the extension's vscode volume, which the extension mounts itself via --mount type=volume,source=vscode,target=/vscode,external=true.
2. Update VS Code to a new commit.
3. Restart VS Code so all windows are restored and reconnect to their containers at the same time.
4. Every window finds the server for the new commit missing and installs it concurrently.

Does this issue occur when you try this locally?: Not applicable - the failing step is the remote server install, which has no local counterpart.

Does this issue occur when you try this locally and all extensions are disabled?: Not tested - the failure occurs in the Dev Containers extension's own server install, before any workspace extension is loaded.

Observed behaviour

At 05:51:50 UTC all six windows ran test -d '/vscode/vscode-server/bin/linux-x64/a44adf...' and got exit 1. Each then downloaded its own copy of the 229 MB tarball in parallel (205-213 s per window, bandwidth split six ways), ran mkdir -p <target>_<epoch-ms>, extracted with tar --no-same-owner -xz -C <tmp> (no --strip-components; the tarball has a single top-level directory vscode-server-linux-x64/), and finished with mv -n <tmp>/* <target> && rmdir <tmp>.

window	temp dir suffix	mv at (UTC)	result
1	_1788846915964	05:55:20.651	ok, created <target>
2	_1788846918442	05:55:33.161	exit 0, but nested copy inside <target>
3	_1788846920205	05:55:42.402	rmdir failed, container start aborted
4	_1788846920203	05:55:45.299	rmdir failed, container start aborted
5	_1788846923742	05:55:51.127	rmdir failed, container start aborted
6	_1788846924122	05:55:57.543	rmdir failed, container start aborted

Inode birth times confirm the nesting: <target> btime 05:55:16.081 (window 1's tar), <target>/vscode-server-linux-x64 btime 05:55:18.556 and ctime 05:55:33.187 (window 2's tar and mv).

The same volume also held a byte-identical nested vscode-server-linux-x64/ copy under two older commit directories (110a328e..., fcf60477...), so the race had already occurred on at least two earlier updates without being noticed.

Log excerpt from one of the failing windows:

[6738 ms] Start: Run in container: test -d '/vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc'
[6765 ms] Exit code 1
[6776 ms] Installing VS Code Server for commit a44adf7f53e00964ab890f9f8758a334f1fc15bc
[6799 ms] Start: Downloading VS Code Server
[220166 ms] Start: Run in container: mkdir -p /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742
[221617 ms] Start: Run in container: (dd iflag=fullblock bs=8192 count=27942; dd iflag=fullblock bs=7228 count=1) | tar --no-same-owner -xz -C /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742
[247487 ms] 228900864 bytes (229 MB, 218 MiB) copied, 25.8351 s, 8.9 MB/s
[247551 ms] Start: Run in container: mv -n /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742/* /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc && rmdir /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742
[247644 ms] rmdir: failed to remove '/vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742': Directory not empty
[247670 ms] Exit code 1
[247710 ms] Start: Run in container: rm -rf /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742
[248515 ms] Command in container failed: mv -n /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742/* /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc && rmdir /vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742
[248536 ms] rmdir: failed to remove '/vscode/vscode-server/bin/linux-x64/a44adf7f53e00964ab890f9f8758a334f1fc15bc_1788846923742': Directory not empty
[248567 ms] Exit code 1

Reopening the failed workspaces later works, because test -d now passes.

Root cause

The server install helper in dist/extension/extension.js of 0.466.0 (minified):

let c = `${r}_${Date.now()}`;
try {
  await t.exec(`mkdir -p ${c}`);
  await t.exec(`${_f(l.byteLength,!0)} | tar --no-same-owner -xz -C ${c}`, {stdin: l, ...});
  await t.exec(`mv -n ${Re.posix.join(c,"*",".")} ${r} && rmdir ${c}`);
} catch (l) { try { await t.exec(`rm -rf ${c}`) } catch {} ; throw l }

• The only existence check (test -d <target>) happens before a multi-minute download; there is no re-check before the mv.
• There is no lock across windows or containers sharing the volume.
• mv -n without -T changes meaning depending on whether the target exists: rename, move-into, or silent skip. Only the first is intended.
• The caller retries only on a corrupt tarball (unexpected end of file|not in gzip format); any other failure aborts the container start.
• The extension's own ~/.vscode-server fallback is not used in this path.

Expected behaviour

Concurrent windows either coordinate (one installs, the others wait), or each install is idempotent: if the target already holds a complete server when the install finishes, the temp dir is discarded and the start continues. In no case should a successful-looking install leave a nested duplicate inside the target.

Suggested fix

Any one of:

1. Take a lock before the download - e.g. atomic mkdir <target>.lock with wait/poll and a stale-lock timeout - so only one window downloads and installs.
2. Extract with --strip-components=1 and finish with mv -T <tmp> <target> (atomic rename that fails if the target exists), treating EEXIST / Directory not empty as "another installer won": rm -rf <tmp> and proceed if <target>/bin/code-server exists.
3. At minimum, re-run test -d <target> immediately before the mv and skip the install if it is present.

Option 1 would also remove the side effect that N windows download the same tarball N times.

Impact

After every VS Code update, with several dev-container windows restored at once, most of them fail to open and have to be reopened manually. The shared volume accumulates roughly 700 MB of dead nested copies per update.

Workaround

Open one dev-container window first, wait for the server install to finish, then open the rest. To clean up an affected volume, remove /vscode/vscode-server/bin/linux-x64/<commit>/vscode-server-linux-x64 and any leftover <commit>_<epoch> directories.

Related issues

• #11781 (open) - the same mv-into-an-existing-directory nesting, but in the WSL extension's wslDownload.sh; different component and trigger.
• #6860 (closed, not planned) - identical mv -n ... && rmdir ... / Directory not empty abort, attributed there to a stale broken symlink rather than a concurrent install.
• #7176 (closed) - server install failure on the shared vscode volume, resolved by manual cleanup.

---

LLM disclosure: the investigation and this text were prepared with Claude (Anthropic) on the reporter's behalf. Log excerpts, timestamps and the code snippet are quoted verbatim from the affected machine.