FileSystemWatcher on macOS: process-wide serialization + blocking fseventsd RPC convoys under watcher churn (seconds-to-minutes stalls)

### Description

On macOS, `FileSystemWatcher` start/stop operations serialize **process-wide** on a static lock in `StaticWatcherRunLoopManager` ([FileSystemWatcher.OSX.cs](https://github.com/dotnet/runtime/blob/v10.0.9/src/libraries/System.IO.FileSystem.Watcher/src/System/IO/FileSystemWatcher.OSX.cs) — `s_lockObject` guarding `ScheduleEventStream`/`UnscheduleFromRunLoop`), and the start path performs a blocking mach RPC to the machine-global `fseventsd` daemon (`FSEventStreamStart → register_with_server → f2d_register_rpc → mach_msg`).

Under watcher churn — many short-lived watchers created and disposed from several threads — these two serialization points convoy: individual `EnableRaisingEvents = true` calls stall for **seconds to over a minute** on an otherwise idle machine. Because `fseventsd` is machine-global, concurrent processes churning watchers amplify each other across process boundaries.

The mainstream way to hit this without ever touching `FileSystemWatcher` directly is integration testing: `Host`/`WebApplication` config uses `reloadOnChange: true` by default (appsettings.json, appsettings.{env}.json, user secrets — ~3 watchers per host), and a `WebApplicationFactory`-per-test suite builds hundreds of hosts per run. In our real suite (~650 hosts/run) the stalls routinely exceeded `Microsoft.AspNetCore.Mvc.Testing`'s 5-minute entry-point wait, producing the widely-reported *"Timed out waiting for the entry point to build the IHost after 00:05:00"* — and worse: after the factory times out and unsubscribes its diagnostic-listener capture, the still-parked entry point eventually finishes building **un-intercepted** and runs the *real* application (real Kestrel, real database connections) inside the test process. Disabling the reload watchers (`hostBuilder:reloadConfigOnChange=false`) took the same suite from 11–35 minutes with 2–41 such timeouts per run to **~20 seconds with zero**, which is what pinned the watchers as the cause.

### Reproduction Steps

Self-contained console app (no ASP.NET involved). 8 threads × 100 iterations; each iteration creates 3 watchers on the thread's own temp directory with `EnableRaisingEvents = true`, then disposes them — mimicking a config-reload host churn.

```xml

Exe
net10.0
enable
enable

```

```csharp
// Program.cs
using System.Collections.Concurrent;
using System.Diagnostics;

int threads = args.Length > 0 ? int.Parse(args[0]) : 8;
int iterations = args.Length > 1 ? int.Parse(args[1]) : 100;
int watchersPerOp = args.Length > 2 ? int.Parse(args[2]) : 3;

var latencies = new ConcurrentBag();
var stalls = new ConcurrentBag();

var workers = Enumerable.Range(0, threads).Select(w => new Thread(() =>
{
var dir = Directory.CreateTempSubdirectory($"fsw-{w}-").FullName;
for (var i = 0; i < iterations; i++)
{
var sw = Stopwatch.StartNew();
var ws = new List();
for (var k = 0; k < watchersPerOp; k++)
{
ws.Add(new FileSystemWatcher(dir, "*.json") { EnableRaisingEvents = true });
}
foreach (var fw in ws) fw.Dispose();
sw.Stop();
latencies.Add(sw.Elapsed.TotalMilliseconds);
if (sw.Elapsed.TotalSeconds >= 1) stalls.Add(sw.Elapsed.TotalSeconds);
}
Directory.Delete(dir, true);
})).ToList();

var total = Stopwatch.StartNew();
workers.ForEach(t => t.Start());
workers.ForEach(t => t.Join());
total.Stop();

var sorted = latencies.OrderBy(x => x).ToArray();
Console.WriteLine($"threads={threads} iterations={iterations} watchersPerOp={watchersPerOp}");
Console.WriteLine($"total={total.Elapsed.TotalSeconds:F1}s ops={sorted.Length}");
Console.WriteLine($"latency ms: p50={sorted[sorted.Length/2]:F0} p90={sorted[(int)(sorted.Length*0.9)]:F0} p99={sorted[(int)(sorted.Length*0.99)]:F0} max={sorted[^1]:F0}");
Console.WriteLine($"ops taking >=1s: {stalls.Count}" + (stalls.Count > 0 ? $" (worst {stalls.Max():F1}s)" : ""));
```

`dotnet run -c Release`

### Expected behavior

Starting and stopping a `FileSystemWatcher` is a lightweight operation; concurrent watcher churn from a few threads should complete in milliseconds per operation, without process-wide serialization or unbounded stalls.

### Actual behavior

On an otherwise idle machine:

```
threads=8 iterations=100 watchersPerOp=3
total=91.3s ops=800
latency ms: p50=337 p90=356 p99=8224 max=68753
ops taking >=1s: 8 (worst 68.8s)
```

A median of **337 ms** to start+stop three watchers, a p99 of **8.2 s**, and a worst case of **68.8 s** for a single iteration. Under additional machine load (e.g. two test runs in parallel — `fseventsd` is machine-global) the tail grows into minutes; native stacks captured with macOS `sample` during a stalled run show the parked threads inside the FSEvents registration RPC (excerpt attached):

```
FSEventStreamStart (in FSEvents)
register_with_server (in FSEvents)
f2d_register_rpc (in FSEvents)
mach_msg / mach_msg2_trap (in libsystem_kernel.dylib)
```

with the queue behind them waiting on `StaticWatcherRunLoopManager.s_lockObject` (and others in `FSEventStreamScheduleWithRunLoop` / `FSEventStreamInvalidate` on the shared run loop).

[sample-stacks-excerpt.txt](https://github.com/user-attachments/files/30351570/sample-stacks-excerpt.txt)

### Regression?

Not believed to be a regression; the shared-run-loop design with the static lock has been in place for years (dotnet/runtime#26577 replaced per-watcher threads with the shared run loop). The impact has grown with the prevalence of `WebApplicationFactory`-per-test integration suites on minimal hosting.

### Known Workarounds

- For hosting-based test suites: pass `hostBuilder:reloadConfigOnChange=false` (host configuration) so test hosts create no reload watchers. This took our ~650-host suite from 11–35 min with 2–41 five-minute IHost-build timeouts per run to ~20 s with zero.
- Generally: avoid short-lived `FileSystemWatcher` churn on macOS.

### Configuration

- .NET SDK 10.0.302, runtime 10.0.10 (`Microsoft.NETCore.App 10.0.10`)
- macOS 26.5.1, arm64 (Apple Silicon, 18 cores)
- Reproduces in-process with the plain console app above; amplification observed across processes (machine-global `fseventsd`)

### Other information

The serialization points are: (1) `StaticWatcherRunLoopManager.ScheduleEventStream` — a single static lock for every watcher schedule/unschedule in the process, held around CFRunLoop bookkeeping, with the first scheduler also blocking on run-loop thread startup; (2) `FSEventStreamStart`'s synchronous registration RPC to `fseventsd`. Possible directions: per-stream `dispatch_queue_t` scheduling via `FSEventStreamSetDispatchQueue` (avoids the shared run loop entirely), and/or moving `FSEventStreamStart` off the lock.

Start in src/libraries/System.IO.FileSystem.Watcher/src/System/IO/FileSystemWatcher.OSX.cs, focusing on StaticWatcherRunLoopManager, ScheduleEventStream, and UnscheduleFromRunLoop. Run the supplied Program.cs reproduction with dotnet run -c Release, then trace the lock and FSEventStreamStart behavior; done should include a validated design and tests or measurements showing watcher churn no longer causes the reported stalls.

Written by the indexing model from the issue text.