First Process-redirected-stdio async read in an NUnit-hosted process throws spurious SocketException(99) EADDRNOTAVAIL on Linux
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
[dotnet-runtime-131915-repro.zip](https://github.com/user-attachments/files/30768040/dotnet-runtime-131915-repro.zip)
## Description
On Linux, the **first** read of a child process's redirected stdout via
`Process.StandardOutput.BaseStream.ReadAsync` fails with:
```
System.Net.Sockets.SocketException (99): Cannot assign requested address
```
`EADDRNOTAVAIL` is an address-assignment error. The descriptor is a pipe — `fstat` reports
`S_IFIFO`, which has no address — so this error cannot legitimately apply to it.
**No syscall ever returns it.** A full unfiltered `strace -f` of a failing run (12 MB,
~130k syscalls, verified to be capturing 27k+ `recvmsg` / 13k+ `read`) contains **zero**
occurrences of `EADDRNOTAVAIL`, and no `recv*`/`read` returning any errno other than `EAGAIN`.
The value is produced inside the managed socket layer.
The path is a pipe being driven through a `Socket`: `PipeStream.ReadAsyncCore` calls
`InternalHandle.PipeSocket.ReceiveAsync(...)`, and `SafePipeHandle.CreatePipeSocket` wraps the
raw pipe fd via `new Socket(new SafeSocketHandle(handle, ownsHandle))`.
It reproduces **100% deterministically** under an NUnit + Microsoft.Testing.Platform host, and
**only** the first such read in the process — every subsequent read is clean. It does **not**
reproduce in a console host, nor under MSTest on the same test platform (see *Other
information* for the full scope matrix).
## Reproduction Steps
Two files in an empty directory. Runs in a few seconds, fails every time.
**`MtpPipeRepro.csproj`**
```xml
net10.0
enable
disable
true
Exe
true
true
```
**`PipeReadTests.cs`**
```csharp
using System.Diagnostics;
using System.Net.Sockets;
using NUnit.Framework;
[TestFixture]
public class PipeReadTests
{
private static string SpawnReadAndCatch()
{
Process p = null;
string failure = null;
try
{
p = Process.Start(new ProcessStartInfo("/bin/cat")
{
RedirectStandardInput = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
});
var reader = new Thread(() =>
{
var buf = new byte[8192];
try
{
while (p.StandardOutput.BaseStream.ReadAsync(buf, 0, buf.Length)
.GetAwaiter().GetResult() > 0) { }
}
catch (IOException ex) when (ex.InnerException is SocketException se)
{
failure = $"{se.SocketErrorCode} ({se.ErrorCode}) {se.Message}";
}
catch (SocketException se)
{
failure = $"{se.SocketErrorCode} ({se.ErrorCode}) {se.Message}";
}
}) { IsBackground = true };
reader.Start();
p.StandardInput.Write(new string('x', 256) + "\n");
p.StandardInput.Flush();
p.StandardInput.Close();
reader.Join(3000);
}
finally
{
try { if (p is { HasExited: false }) p.Kill(true); } catch { }
p?.Dispose();
}
return failure;
}
// Fails every run.
[Test, Order(1)]
public void FirstRedirectedStdioAsyncRead_ThrowsSpuriousAddressNotAvailable()
{
var failure = SpawnReadAndCatch();
TestContext.Out.WriteLine($"first spawn -> {failure ?? "(no failure)"}");
Assert.That(failure, Is.Null);
}
// Control: passes every run, showing this is one-time initialisation, not a race.
[Test, Order(2)]
public void SubsequentReads_AreClean()
{
var failures = new List();
for (int i = 0; i < 60; i++)
if (SpawnReadAndCatch() is { } f) failures.Add($"[{i}] {f}");
TestContext.Out.WriteLine($"subsequent spawns=60 failures={failures.Count}");
Assert.That(failures, Is.Empty);
}
}
```
```bash
dotnet build
./bin/Debug/net10.0/MtpPipeRepro
```
> **Note:** `true` is required. Without it the project
> builds and the apphost exits 0 without running any test, which looks like a pass.
## Expected behavior
`ReadAsync` on a `Process`-redirected stdout stream either succeeds, or fails with an error that
is meaningful for a pipe (e.g. `EPIPE`/`Shutdown`, EOF as a 0-length read). `EADDRNOTAVAIL`
cannot apply to a pipe and should never surface from this API.
## Actual behavior
The first such read throws. Output, identical on 6/6 runs:
```
first spawn -> AddressNotAvailable (99) Cannot assign requested address
subsequent spawns=60 failures=0
```
Stack, captured from a real failure:
```
System.Net.Sockets.SocketException (99): Cannot assign requested address
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ReceiveAsync(Socket socket, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.ReceiveAsync(Memory`1 buffer, SocketFlags socketFlags, Boolean fromNetworkStream, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.ReceiveAsync(Memory`1 buffer, SocketFlags socketFlags, CancellationToken cancellationToken)
at System.IO.Pipes.PipeStream.ReadAsyncCore(Memory`1 destination, CancellationToken cancellationToken)
at System.Threading.Tasks.Task`1.InnerInvoke()
at System.Threading.Thread.StartCallback()
```
`AwaitableSocketAsyncEventArgs.CreateException` only materialises a `SocketError` already stored
on the `SocketAsyncEventArgs`, so the incorrect value is assigned when the async receive
**completes** — consistent with no failing syscall existing.
## Regression?
**Unknown.** Only `Microsoft.NETCore.App 10.0.10` is installed on the investigation host; 8.x and
9.x were not tested. Happy to test other runtimes if that would help.
## Known Workarounds
Two, both used successfully:
1. **Warm-up (preferred).** Because it is strictly first-use, spawning one throwaway child with
redirected stdio at startup and reading once absorbs the failure; everything afterwards is
clean. Supported by the control test above (spawn 0 fails, spawns 1–60 always pass).
2. **Bounded retry** around the operation. Effective, but note it **orphans the failed
attempt's child process** — the throwing call yields no handle to dispose. Measured by
sampling process counts during runs: every retry peaks at 2 concurrent children instead of 1.
## Configuration
| | |
|---|---|
| .NET SDK | 10.0.302 |
| Runtime | Microsoft.NETCore.App 10.0.10 |
| OS | Ubuntu 24.04.4 LTS |
| Kernel | 6.8.0-136-generic |
| Architecture | x64 |
| NUnit / NUnit3TestAdapter | 4.4.0 / 6.0.0-beta.2.1 |
| Microsoft.NET.Test.Sdk | 18.0.1 |
## Other information
**It is first-use, not a race.** Initially mistaken for a concurrency race. The failure attaches
to whichever test spawns first, always at spawn index 0:
| Ordering | Result |
|---|---|
| Concurrent test first | concurrent takes the failure; sequential clean |
| Sequential test first (`Order(1)`) | sequential takes the failure; concurrent clean |
| Instrumented spawn index | `FAILED_AT_SPAWN_INDEX=0`, 5/5 runs |
**Scope — what does and does not reproduce:**
| Host | Spawns | Failures |
|---|---|---|
| Console app, plain | 1,152 | 0 |
| Console app + `Console.SetOut`/`SetError` redirected | 1,152 | 0 |
| Console app, 24 & 64 concurrent workers + socket churn | 1,832 | 0 |
| MSTest + Microsoft.Testing.Platform | 1,536 | 0 |
| **NUnit + Microsoft.Testing.Platform** | 1/run | **1 — every run** |
Swapping only the test framework removes it, so the trigger appears specific to the
NUnit + NUnit3TestAdapter stack rather than Microsoft.Testing.Platform. I have not separated
`nunit/nunit` from `nunit/nunit3-vs-adapter`, and only one alternative framework was tested, so
this does not establish exclusivity. Filing here rather than with NUnit because nothing a test
framework legitimately does — installing a `SynchronizationContext`, flowing `AsyncLocal`
context, capturing output — should make `PipeStream.ReadAsync` return an address error on a pipe.
**Observation sensitivity** (worth knowing before instrumenting): under `strace` the failure rate
in the original project roughly doubled, while injecting a `DOTNET_STARTUP_HOOKS` assembly with a
`FirstChanceException` handler suppressed it entirely (0/10 against a ~30% baseline). The
deterministic repro above is not affected by this.
**Possibly related:** #130577 — same `Cannot assign requested address`, same .NET 10 / Linux, but
Unix domain sockets in MSBuild's out-of-proc node IPC rather than anonymous pipes, and there the
SocketExceptions precede a CLR fatal error. Different transport, same impossible errno from the
same Unix socket layer; may share a root cause. The repro here is considerably smaller if so.
**Ruled out** (measured, not assumed): kernel error; ThreadPool starvation
(`DOTNET_ThreadPool_MinThreads=64` → no improvement); socket-engine contention
(`DOTNET_SYSTEM_NET_SOCKETS_THREAD_COUNT=1`, 16 → 1 → no worsening); resource exhaustion (no
leaked processes/fds, 55 GiB RAM free, `/dev/shm` at 1%); CPU load (no clean correlation).
**Not determined:** which assignment sets `SocketError.AddressNotAvailable` on the
`SocketAsyncEventArgs`, and how errno 99 enters managed code at all given no syscall returns it.
---
*Suggested area: `area-System.Net.Sockets` (or `area-System.IO.Pipes`).*
Contributor guide
Research direction
Start with MtpPipeRepro.csproj and PipeReadTests.cs, then run dotnet build and ./bin/Debug/net10.0/MtpPipeRepro on Linux to confirm the first-read failure. Trace the named PipeStream.ReadAsyncCore, InternalHandle.PipeSocket.ReceiveAsync, SafePipeHandle.CreatePipeSocket, and Socket.AwaitableSocketAsyncEventArgs paths. Done means identifying the managed source of the spurious error, fixing it without breaking pipe reads, and adding a regression test for the NUnit-hosted first read.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, linux
- Domain
- networking, operating-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100