dotnet / dotnet/sdk

File-based apps fail when executed concurrently due to shared build cache contention

Open
#52,773 11 comments 3 reactions 0 assignees View on GitHub
Area-run-file untriaged
Dominant language
C#
Stars
3.2k
Forks
1.3k
PR merge metrics
PR metrics pending

Description

### Describe the bug

File-based apps (single `.cs` files executed via `dotnet run file.cs` or `dotnet file.cs`) cannot be run concurrently. When multiple instances of the same file-based app are started simultaneously, they compete for the same build cache directory and output files, causing random build failures with file access errors.

This occurs because all invocations of the same file-based app presumably share a single build cache directory based on the file hash, and multiple processes attempt to write to the same DLL and cache files simultaneously.

### To Reproduce

1. Create a simple file-based app (`concurrent-test.cs`):

```csharp
#!/usr/bin/env dotnet
#:property PublishAot=false

var id = args.Length > 0 ? args[0] : "0";
var delay = args.Length > 1 ? int.Parse(args[1]) : 100;

Console.WriteLine($"[{DateTime.Now:HH:mm:ss.fff}] Process {id} starting (PID: {Environment.ProcessId})...");
Thread.Sleep(delay);
Console.WriteLine($"[{DateTime.Now:HH:mm:ss.fff}] Process {id} completed!");
```

2. Clear the build cache (important - the bug only occurs on initial compilation):

```bash
# macOS
rm -rf ~/Library/Application\ Support/dotnet/runfile/concurrent-test-*

# Linux
rm -rf ~/.config/dotnet/runfile/concurrent-test-*
```

3. Run 20 concurrent instances:

```bash
seq 1 20 | xargs -P 20 -I {} dotnet concurrent-test.cs {} 200
```

**Reproduction repository:** https://github.com/alexaka1/repro-dotnet--sdk-52773

### Exceptions (if any)

Multiple processes fail with errors like:

```
CSC : error CS2012: Cannot open '/Users/.../dotnet/runfile/concurrent-test-/obj/debug/concurrent-test.dll'
for writing -- The process cannot access the file '...concurrent-test.dll' because it is being used by another process.

The process cannot access the file '/Users/.../dotnet/runfile/concurrent-test-/build-success.cache'
because it is being used by another process.
```

The error occurs because all 20 processes share the same cache directory and compete for:

- `concurrent-test.dll` (the compiled output)
- `build-success.cache` (build status file)

### Expected behavior

All 20 processes should either:

1. Compile independently into separate directories (per-process isolation), OR
2. Coordinate access to a shared cache with proper file locking, OR
3. Reuse a cached binary safely after the first successful build

### Actual behavior

Random processes fail with file access contention errors during compilation. The failure rate increases with higher concurrency (20+ instances).

### Workarounds

- run file-based apps sequentially instead of in parallel
- run `dotnet build app.cs` on your script first, and then the SDK find the cached build and won't run into concurrency issues

### Further technical details

**Cache directory structure:**

- **macOS:** `~/Library/Application Support/dotnet/runfile//`
- **Linux:** `~/.config/dotnet/runfile//`
- **Windows:** `%LOCALAPPDATA%\dotnet\runfile\\`

**Tested with:**

- `dotnet concurrent-test.cs`
- `dotnet run --file concurrent-test.cs`
- `dotnet run --file concurrent-test.cs --no-cache`

All variations exhibit the same race condition because they share the same underlying build output directory.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.