Console.SetOut can deadlock with writes through a previously captured Console.Out on Unix
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
## Description
On Unix, writing concurrently through the current `Console.Out` and a previously
captured `Console.Out` can deadlock after `Console.SetOut` redirects output to a
writer that forwards to the original writer.
The deadlock is caused by a lock-order inversion between two synchronized
`TextWriter` instances:
- **O** is the original `Console.Out`.
- **R** is the current `Console.Out`, installed by `Console.SetOut`.
The two write paths acquire these writers in opposite orders:
1. Writing through **R** holds **R** in `SyncTextWriter.WriteLine`, then the
forwarding writer writes to **O**: **R → O**.
2. Writing directly through **O** holds **O** in `SyncTextWriter.WriteLine`,
then `UnixConsoleStream` reaches `ConsolePal.WriteFromConsoleStream`, which
locks the mutable, current `Console.Out` (**R**): **O → R**.
If these paths execute concurrently, each thread can wait forever for the
writer owned by the other thread.
This was originally encountered as an intermittent deadlock in
https://github.com/dotnet/macios/issues/26537. NUnit redirected `Console.Out`
to a capture writer that forwards off-context output to the original writer.
An asynchronous callback wrote through the redirected writer while the test
runner's event-pump thread wrote through the saved original writer. Neither
NUnit's forwarding writer nor the test runner explicitly locked these writers;
the conflicting locks came from `SyncTextWriter` and `ConsolePal`.
## Reproduction
Create a .NET 10 console application and replace `Program.cs` with:
```csharp
using System.Text;
// Keep this true for a deterministic reproduction. Change it to false to
// reproduce the same deadlock naturally, without explicitly locking either
// writer.
bool useDeterministicReproduction = true;
TextWriter originalOut = Console.Out;
Console.WriteLine ($"Starting the {(useDeterministicReproduction ? "deterministic" : "racy")} reproduction.");
Console.SetOut (new ForwardingTextWriter (originalOut));
TextWriter redirectedOut = Console.Out;
if (useDeterministicReproduction)
RunDeterministicReproduction ();
else
RunRacyReproduction ();
void RunDeterministicReproduction ()
{
using var redirectedWriterLocked = new ManualResetEventSlim ();
using var originalWriterLocked = new ManualResetEventSlim ();
var redirectedWriterThread = new Thread (() => {
lock (redirectedOut) {
redirectedWriterLocked.Set ();
originalWriterLocked.Wait ();
redirectedOut.WriteLine ("redirected writer");
}
});
var originalWriterThread = new Thread (() => {
redirectedWriterLocked.Wait ();
lock (originalOut) {
originalWriterLocked.Set ();
originalOut.WriteLine ("original writer");
}
});
redirectedWriterThread.Start ();
originalWriterThread.Start ();
redirectedWriterThread.Join ();
originalWriterThread.Join ();
}
void RunRacyReproduction ()
{
var redirectedWriterThread = new Thread (() => {
while (true) {
Console.WriteLine ("redirected writer");
Thread.Yield ();
}
}) {
IsBackground = true,
};
var originalWriterThread = new Thread (() => {
while (true) {
originalOut.WriteLine ("original writer");
Thread.Yield ();
}
}) {
IsBackground = true,
};
redirectedWriterThread.Start ();
originalWriterThread.Start ();
redirectedWriterThread.Join ();
originalWriterThread.Join ();
}
sealed class ForwardingTextWriter : TextWriter {
readonly TextWriter writer;
public ForwardingTextWriter (TextWriter writer)
{
this.writer = writer;
}
public override Encoding Encoding => writer.Encoding;
public override void Write (char value)
{
writer.Write (value);
}
public override void Write (string? value)
{
writer.Write (value);
}
public override void WriteLine (string? value)
{
writer.WriteLine (value);
}
}
```
Run:
```shell
dotnet run -c Release
```
The default path prints its initial line and then deterministically deadlocks.
The explicit `lock` statements in this path only make the existing lock-order
inversion 100% reproducible. They acquire the same writer monitors that the
subsequent `WriteLine` calls acquire implicitly, and those acquisitions are
reentrant. The events ensure that each thread owns its first writer before
either requests the second.
To reproduce without explicitly locking either writer, change:
```csharp
bool useDeterministicReproduction = false;
```
The two background threads then repeatedly exercise the two real write paths
until they race in the wrong order. On my machine this mode deadlocks quickly;
in one run it stopped making progress after producing 1,074 bytes of output.
## Expected behavior
Concurrent writes through the current `Console.Out` and a previously obtained
`Console.Out` should not deadlock.
In particular, a console stream obtained before `Console.SetOut` should not
later synchronize on an unrelated writer installed as the current
`Console.Out`.
## Actual behavior
Both threads remain blocked in monitor acquisition:
- The redirected-writer thread owns **R** and waits for **O**.
- The original-writer thread owns **O** and waits for **R** in
`ConsolePal.WriteFromConsoleStream`.
Attaching LLDB shows both threads waiting below `Monitor_Enter_Slowpath`.
## Runtime implementation
`Console.SetOut` synchronizes the supplied writer:
```csharp
if (newOut != TextWriter.Null)
{
newOut = TextWriter.Synchronized(newOut);
}
```
`SyncTextWriter.Write*` methods use `MethodImplOptions.Synchronized`, so a write
through either **O** or **R** holds that writer's monitor.
On Unix, the original writer eventually reaches:
```csharp
internal static unsafe void WriteFromConsoleStream(
SafeFileHandle fd,
ReadOnlySpan buffer)
{
EnsureConsoleInitialized();
lock (Console.Out)
{
Write(fd, buffer);
}
}
```
The problematic part is that an existing console stream synchronizes on the
mutable current `Console.Out`, rather than on a stable lock associated with the
stream or console state.
Git history indicates that `WriteFromConsoleStream` and this
`lock (Console.Out)` were introduced in commit
`ec0251bf15d18aaffff01741e032bd197670f473`, as part of
https://github.com/dotnet/runtime/pull/94414.
## Environment
Reproduced with:
```text
.NET SDK: 10.0.302
SDK commit: 35b593bebf
Runtime: Microsoft.NETCore.App 10.0.10
RID: osx-arm64
Operating system: macOS 26.6.2 (25G83)
Architecture: arm64
```
The affected implementation is Unix-specific; I have not reproduced or
investigated this on Windows.
Contributor guide
Research direction
Start with the UnixConsoleStream path and ConsolePal.WriteFromConsoleStream, then compare its lock on Console.Out with the synchronization applied by Console.SetOut and SyncTextWriter.WriteLine. Use the provided Program.cs reproduction and run `dotnet run -c Release`; done means concurrent writes through the current and previously captured writers complete without deadlocking on Unix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100