Task.ScheduleAndStart completes a task that is still queued when worker-thread creation fails, causing an uncatchable InvalidCastException in RunContinuations

### Description

When `ThreadPoolTaskScheduler.QueueTask` throws *after* the work item has already been committed to the thread-pool queue, `Task.ScheduleAndStart` catches the exception and marks that task completed. The task is still in the queue, so a live pool worker dequeues and executes it, completing it a second time. The second completion installs and then re-reads the completion sentinel, and crashes with `InvalidCastException` inside `Task.RunContinuations`.

In practice the trigger is `PortableThreadPool.WorkerThread.CreateWorkerThread` failing to start a thread under memory pressure.

**This appears to be the unfixed sibling of [PR #115659](https://github.com/dotnet/runtime/pull/115659).** That PR fixed a double-`Finish` reached through `TryRunInline`, described there as: *"the code would incorrectly call `Finish` in the catch block … This set a sentinel in the continuation object, which could later cause `ExecuteWithThreadLocal` to invoke `Finish` again, triggering an `InvalidCastException`."* Its author noted the fix might address only part of the broader problem. This report is a second entry point into the same sentinel mechanism which that PR does not touch — reached through `ScheduleAndStart`'s own catch rather than through inlining — together with a deterministic reproduction, which prior reports of this symptom have lacked.

The severity is not simply "another error during resource exhaustion". The first exception is **catchable** at the scheduling call site, and an application that handles it correctly stays healthy; the second is **uncatchable** and always fatal. Details under *Other information*.

### Reproduction Steps

Deterministic; no memory pressure required. Save as `repro.cs` and run `dotnet run repro.cs` on .NET 10.

Reflection is used only to stand in for two runtime-internal steps, not to manufacture the outcome — `ThreadPoolTaskScheduler`, `ExecuteFromThreadPool` and `UnsafeQueueUserWorkItemInternal` are all internal, and `Task` exposes no public execute path. The single reflection call enqueues the task exactly as `ThreadPoolTaskScheduler.QueueTask` does for a non-`LongRunning` task; **a genuine pool worker then executes it.**

```csharp
using System.Reflection;

var task = new Task(() => Console.WriteLine("[task] body ran"));

// Step 1: scheduling fails. In production this is CreateWorkerThread throwing OOM
// AFTER ThreadPoolWorkQueue.Enqueue already committed the item to the queue.
// ScheduleAndStart catches, marks the task finished, and rethrows to us.
try
{
task.Start(new FailingScheduler());
}
catch (TaskSchedulerException ex)
{
Console.WriteLine($"[app] caught {ex.GetType().Name} / inner {ex.InnerException?.GetType().Name} -- handled, continuing");
}

Console.WriteLine($"[app] task status={task.Status} isCompleted={task.IsCompleted}");

// Step 2: the item was already queued in production, so enqueue it the way
// ThreadPoolTaskScheduler.QueueTask does for a non-LongRunning task. A REAL pool
// worker then dequeues and executes the already-completed task.
var enqueue = typeof(ThreadPool).GetMethod(
"UnsafeQueueUserWorkItemInternal",
BindingFlags.Static | BindingFlags.NonPublic);
enqueue!.Invoke(null, [task, false]);

Console.WriteLine("[app] enqueued; app is healthy and waiting...");
Thread.Sleep(4000);
Console.WriteLine("[app] REACHED THE END -- bug did not reproduce");

sealed class FailingScheduler : TaskScheduler
{
protected override void QueueTask(Task task) =>
throw new OutOfMemoryException("stands in for the OOM thrown by CreateWorkerThread");

protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued) => false;

protected override IEnumerable GetScheduledTasks() => [];
}
```

Note a custom `TaskScheduler` cannot reproduce this on its own: a custom scheduler's tasks run through `TaskScheduler.TryExecuteTask` → the guarded `Task.ExecuteEntry`, which refuses an already-completed task and returns `false`. Only the built-in pool's dispatch reaches the unguarded `ExecuteEntryUnsafe`. That is why step 2 enqueues to the real pool.

### Expected behavior

A failure to provision a worker thread should leave the application with a *handleable* scheduling failure and consistent task state. Concretely, either:

- the task is not completed by `ScheduleAndStart` when it is still queued and will therefore run; or
- the worker-provisioning failure does not escape into the caller's scheduling call at all; or
- the pool's dispatch declines to run a task that has already completed, as `ExecuteEntry` already does.

In all three cases the process survives, and the only exception the application sees is the `TaskSchedulerException` it can catch at the call site.

### Actual behavior

The task is completed twice. The second completion throws `InvalidCastException` on a thread-pool worker with no handler above it, and the process terminates.

Repro output, verbatim (.NET 10.0.11, Windows x64), exit code `0xE0434352`:

```
[app] caught TaskSchedulerException / inner OutOfMemoryException -- handled, continuing
[app] task status=Faulted isCompleted=True
[app] enqueued; app is healthy and waiting...
Unhandled exception. System.InvalidCastException: Unable to cast object of type 'System.Object' to type 'System.Collections.Generic.List`1[System.Object]'.
at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
at System.Threading.Tasks.Task.FinishSlow(Boolean userDelegateExecute)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
at System.Threading.Thread.StartCallback()
```

Two details: the application **handled** the scheduling failure and reported itself healthy before dying, and `[task] body ran` never prints — the task's delegate is not executed at all, so the crash is entirely in the completion tail.

### The same pair observed in production

A long-running .NET 10 service on Windows Server x64, during a host-wide memory shortage. Two unhandled exceptions, 0.94 s apart, in the same process.

First:

```
System.Threading.Tasks.TaskSchedulerException: An exception was thrown by a TaskScheduler.
---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Threading.Thread.StartInternal(ThreadHandle t, Int32 stackSize, Int32 priority, Char* pThreadName)
at System.Threading.Thread.StartCore()
at System.Threading.PortableThreadPool.WorkerThread.CreateWorkerThread()
at System.Threading.PortableThreadPool.WorkerThread.MaybeAddWorkingWorker(PortableThreadPool threadPoolInstance)
at System.Threading.Tasks.Task.ScheduleAndStart(Boolean needsProtection)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ScheduleAndStart(Boolean needsProtection)
at System.Net.Http.HttpConnectionPool.CleanCacheAndDisposeIfUnused()
at System.Net.Http.HttpConnectionPoolManager.RemoveStalePools()
at System.Net.Http.HttpConnectionPoolManager.<>c.<.ctor>b__11_0(Object s)
at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool)
at System.Threading.TimerQueue.FireNextTimers()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
```

Then, 0.94 s later:

```
System.InvalidCastException: Unable to cast object of type 'System.Object' to type 'System.Collections.Generic.List`1[System.Object]'.
at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
at System.Threading.Tasks.Task.FinishSlow(Boolean userDelegateExecute)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
```

Both stacks contain only BCL frames. The caller in the first trace is incidental — routine `HttpClient` connection-pool maintenance firing from a timer; any code scheduling a task in that window would produce the same thing. Because it *is* a timer callback, that first exception also had no handler above it, so in this particular incident the process was already terminating. The severity argument therefore rests on the repro and on the ordinary case where the scheduling caller handles the failure, not on this trace.

### Regression?

**Yes — a regression in .NET 8.**

[PR #74791](https://github.com/dotnet/runtime/pull/74791) (merged September 2022, fixing [#13437](https://github.com/dotnet/runtime/issues/13437)) removed `TryCreateWorkerThread`'s `catch (ThreadStartException)` / `catch (OutOfMemoryException)` and its thread-count rollback, renaming it to a void `CreateWorkerThread`. Before that change the portable pool swallowed worker-start failure, so the exception never reached `ScheduleAndStart` and the task was never completed while queued.

Worth noting the same PR chose `Environment.FailFast()` for **gate**-thread creation failure specifically *"so user code can't catch the exception and possibly lead to a stall"*. The worker-thread path was left propagating, which is what exposes this.

I have not bisected on .NET 6/7 directly; this is from reading that PR's diff against the current source.

### Known Workarounds

**None from application code.** All three points where you would intervene are out of reach:

- the enqueue has already happened when the exception surfaces, so nothing the caller does prevents the second execution;
- the `InvalidCastException` arrives on a pool worker with no handler above it, so no `try`/`catch` can see it;
- the .NET 9+ global handler is also bypassed — see *Other information*.

Only environmental mitigation is available: keeping enough memory headroom (and commit-charge headroom on Windows) that worker-thread creation does not fail. Pre-warming the pool via `ThreadPool.SetMinThreads` may reduce how often a new worker must be created at an unlucky moment, but it does not close the window and I have not measured it.

### Configuration

- **Repro verified:** .NET 10.0.11, Windows 10.0.19045, x64.
- **Production occurrence:** .NET 10, Windows Server, x64, long-running service process.
- **Believed affected:** .NET 8 and later, per *Regression?* above. Source quotes below are from the **v10.0.11** tag; `main` is unchanged on both halves of the mechanism, though it has renamed some members (`EnsureThreadRequested` → `ThreadPool.EnsureWorkerRequested`, dispatch via `Task.ExecuteDirectly`).
- **Not specific to a configuration.** Nothing here is architecture- or OS-specific; it needs only a thread-creation failure while at least one pool worker is alive to drain the queue. Windows x64 is simply where it was seen.

### Other information

### Mechanism

1. `Task.ScheduleAndStart` calls `ThreadPoolTaskScheduler.QueueTask`. For a non-`LongRunning` task that is `ThreadPool.UnsafeQueueUserWorkItemInternal(task, ...)`.

2. `ThreadPoolWorkQueue.Enqueue` **commits the item to the queue first**, and only then asks for a worker:

```csharp
public void Enqueue(object callback, bool forceGlobal)
{
// ... queueing logic that adds callback to various queues ...
EnsureThreadRequested();
}
```

```csharp
internal void EnsureThreadRequested()
{
if (Interlocked.Exchange(ref _separated._hasOutstandingThreadRequest, 1) == 0)
{
ThreadPool.RequestWorkerThread();
}
}
```

3. `RequestWorkerThread` → `MaybeAddWorkingWorker` → `CreateWorkerThread` → `Thread.StartCore` → `Thread.StartInternal` throws, because a stack cannot be allocated for the new thread.

4. That exception unwinds back through `Enqueue` and `QueueTask` into `ScheduleAndStart`'s catch, which records a `TaskSchedulerException`, **calls `Finish` on the task** — installing `s_taskCompletionSentinel` in `m_continuationObject` — and rethrows. The repro proves this transition: `status=Faulted isCompleted=True` immediately after `Start` threw.

5. **The work item is still in the queue.** A running worker dequeues it: `ThreadPoolWorkQueue.Dispatch` → `DispatchWorkItem` → `task.ExecuteFromThreadPool` → `ExecuteEntryUnsafe` → `ExecuteWithThreadLocal` → `Finish` → `FinishSlow` → `FinishContinuations`.

6. `FinishContinuations` does `Interlocked.Exchange(ref m_continuationObject, s_taskCompletionSentinel)` and gets **the sentinel back** from step 4. Being non-null it is passed to `RunContinuations`, where every type test fails and the trailing `(List)continuationObject` cast throws.

### Why the existing guard does not apply

`Task.ExecuteEntry` — the path `TaskScheduler.TryExecuteTask` uses — atomically refuses to run an already-invoked task; called a second time it returns `false`. The pool does not go through it. `Task` does **not** implement `IThreadPoolWorkItem` (`public class Task : IAsyncResult, IDisposable`); `ThreadPoolWorkQueue.DispatchWorkItem` pattern-matches the type and calls `ExecuteFromThreadPool` → `ExecuteEntryUnsafe`, which omits the check on the assumption that a queued task cannot already have completed. Step 4 breaks that assumption.

### Why the second exception is uncatchable

`DispatchWorkItem` puts the global-handler filter only on the *non-*`Task` branch:

```csharp
if (workItem is Task task)
{
// Task workitems catch their exceptions for later observation
// We do not need to pass unhandled ones to ExceptionHandling.s_handler
task.ExecuteFromThreadPool(currentThread);
}
else
{
Debug.Assert(workItem is IThreadPoolWorkItem);
try
{
Unsafe.As(workItem).Execute();
}
catch (Exception ex) when (ExceptionHandling.IsHandledByGlobalHandler(ex))
{
// the handler returned "true" means the exception is now "handled" and we should continue.
}
}
```

So `ExceptionHandling.SetUnhandledExceptionHandler` (.NET 9+) does not see it either. The stated assumption — that task work items handle their own exceptions — is exactly what fails here, because the throw is in the completion machinery rather than in user code.

### The `ContinueWith` path is also exposed, with no catchable signal at all

`ContinueWithTaskContinuation.Run` swallows the first exception by design, and its comment records the assumption that is wrong here:

```csharp
try { continuationTask.ScheduleAndStart(needsProtection: true); }
catch (TaskSchedulerException)
{
// No further action is necessary -- ScheduleAndStart() already transitioned the
// task to faulted. But we want to make sure that no exception is thrown from here.
}
```

Transitioning the task to faulted is treated as sufficient. It is not, because the task is also still queued. On this path the application gets **no** observable antecedent — only the fatal crash.

### Trigger conditions, and three paths that are not exposed

The general condition is any exception escaping `QueueTask` after the item is committed. In practice that is worker-thread provisioning failing, which can come from physical memory exhaustion, commit-charge exhaustion on Windows (fails thread creation even with physical RAM available), a per-process cap such as a container limit or Job object, or thread/handle limits.

Not exposed, which may help scope a fix:

- **`LongRunning` tasks** — `QueueTask` starts a dedicated thread *instead of* enqueueing, so a failure leaves the task un-queued and completing it is correct.
- **An allocation failure inside the enqueue itself** (segment growth, work-stealing array resize) — the item is not committed yet.
- **Custom `TaskScheduler` implementations** — guarded `ExecuteEntry`, as above.

### Proven vs. inferred

- **Proven by the repro:** that `ScheduleAndStart`'s failure path leaves the task completed; that a real pool worker executing that task produces this exception at these frames; and that it is unhandled and fatal while the application had itself handled the scheduling failure.
- **Proven by source inspection** (quoted above): enqueue-before-worker-request ordering; no global-handler filter on the `Task` dispatch branch; the `ContinueWith` swallow.
- **Inferred, in the production incident only:** that the crash 0.94 s after the OOM involved *that same task*. The crash carries independent weight regardless — `m_continuationObject` holds a bare `System.Object` in exactly one state (`s_taskCompletionSentinel`, commented *"m_continuationObject is set to this when the task completes"*), so the `InvalidCastException` alone proves that *some* task's `FinishContinuations` ran twice. Only the cause of that double completion is inferred.
- **Not reproduced:** the OOM half itself. That needs `Thread.StartCore` to fail while a worker is alive to drain the queue — timing- and environment-dependent, which is likely why prior reports describe the symptom as random.

### Related issues

- **[#83520](https://github.com/dotnet/runtime/issues/83520)** — closest prior art: same exception and message, same frames, on CoreCLR, with a reliable repro (PLINQ + `Thread.Interrupt`). Closed as a duplicate of #114262, noting #115659 as the applied fix and that it could be re-opened if the symptom persisted. This may be that residual path.
- **[#26363](https://github.com/dotnet/runtime/issues/26363)** — reported the same sentinel-cast symptom in 2018 and was closed on the reasoning that Task's state transitions guarantee `FinishContinuations` runs only once. The repro above is a deterministic counterexample to that specific invariant.
- **[#114262](https://github.com/dotnet/runtime/issues/114262)** — currently holds this symptom as a dedup target, but is a Mono/iOS report (its text is Mono's *"Specified cast is not valid."*). Filed separately because this analysis is CoreCLR-specific and turns on `ThreadPoolWorkQueue.Enqueue` ordering; happy to be merged if judged the same.
- **[#13062](https://github.com/dotnet/runtime/issues/13062)**, **[#11309](https://github.com/dotnet/runtime/issues/11309)**, **[dotnet/wcf#1679](https://github.com/dotnet/wcf/issues/1679)** — cited only as evidence that thread-creation OOM reaches task scheduling in real deployments and CI. They do **not** show this window: where their stacks have a frame under `QueueTask` it is `Thread.Start`, i.e. the `LongRunning` branch, one of the immune paths above.

### Possible fix directions

Offered tentatively:

- Have `ScheduleAndStart` avoid completing the task when the failure arose *after* the item was committed. Distinguishing that inside `QueueTask` is the hard part.
- Restore something like the rollback PR #74791 removed, so worker-provisioning failure does not reach the caller's scheduling call. That PR already concluded user code should not be able to catch the analogous gate-thread failure; the worker path is arguably the same question answered differently. A naive swallow is not sufficient on its own: `_hasOutstandingThreadRequest` is already set and `MaybeAddWorkingWorker`'s count CAS already committed, so later enqueues will not re-request a worker — that state needs unwinding too.
- Make dispatch tolerate an already-completed task rather than double-completing it, aligning `ExecuteEntryUnsafe` with `ExecuteEntry`'s guarantee. This is closest to what #115659 did for the inlining path.

Trace Task.ScheduleAndStart through ThreadPoolTaskScheduler.QueueTask, ThreadPoolWorkQueue.Enqueue, RequestWorkerThread, MaybeAddWorkingWorker, and CreateWorkerThread, then inspect the second completion in Task.RunContinuations and ExecuteWithThreadLocal. Run the deterministic repro on .NET 10 and verify that a worker-thread creation failure leaves consistent task state without an unhandled InvalidCastException.

Written by the indexing model from the issue text.