.NET: [Bug]: StreamingRun.WatchStreamAsync() can silently stop yielding events after a long-running executor turn completes successfully

Describe the bug

StreamingRun.WatchStreamAsync()'s async-enumerable can silently stop yielding events — no exception, no further events, the await foreach just never sees another item — after a sufficiently long-running executor turn, even though that turn itself completed successfully. MAF never emits that state's own ExecutorCompletedEvent/SuperStepCompletedEvent (or any subsequent event), so a consumer that pumps the stream to detect run completion has no signal to distinguish "the run legitimately finished but MAF forgot to say so" from "the run is still working and just hasn't produced an event yet." There is no WorkflowFailedEvent, no RequestInfoEvent, nothing — the stream is simply quiet forever.

To Reproduce

Reproduced live twice against a real multi-state WorkflowBuilder graph with real (oMLX-backed, OpenAI-compatible) IChatClient executors, package version Microsoft.Agents.AI.Workflows 1.13.0:

1. git-history state (two executor turns observed in the trace, then the stream goes silent):

   [STG-DIAG] RunAsync entered timeoutSpec='15m'
   [STG-DIAG] ... WhenAny returned at elapsed=00:01:16.6, winner=workTask
   [STG-DIAG] workTask completed normally
   [STG-DIAG] RunAsync entered timeoutSpec='15m'
   [STG-DIAG] ... WhenAny returned at elapsed=00:10:00.9, winner=workTask
   [STG-DIAG] workTask completed normally
                                                       <-- WatchStreamAsync's enumerable goes silent here
   

   (STG-DIAG traces bracket our own executor's work — the turn itself completed cleanly, well inside its own caller-side deadline. No further MAF event of any kind followed.)

2. performance state (a pooled analyst dimension completing a real ~10-minute turn cleanly), same shape: the stream ends with zero further events after that turn.

We instrumented and traced across our own call stack (a custom Executor, our pooling/reuse layer, and our WorkflowRunner wrapper around InProcessExecution.StreamAsync(...)/WatchStreamAsync()) and were able to rule out every layer of our code as the cause — the turn's own work completes, control simply never returns from MAF's side afterward, with no error and no further event. We also tried (and reverted) a fix that called StreamingRun.GetStatusAsync() after the stream ended to distinguish "genuinely stalled" from "legitimately finished" — see the "Ask" section below for why that didn't hold up.

We were not able to produce a minimal repro isolated from our own executors/tools, since the defect appears to correlate with turn duration (single-digit minutes) rather than any specific executor behavior we can identify — happy to help narrow it down further if a MAF maintainer has a hypothesis to test against (e.g. an internal timeout/cancellation racing the stream write, or a checkpoint/dispatch loop exiting once its work queue looks empty without checking whether more supersteps are pending).

Expected Behavior

One of:

• The stream keeps yielding events (including, at minimum, a terminal WorkflowCompletedEvent/WorkflowOutputEvent or a WorkflowFailedEvent) until the workflow genuinely reaches a terminal state, or
• If MAF internally decides the run is done/stuck, it emits some event saying so — the consumer should never be left inferring completion from silence alone.

Actual Behavior

The await foreach over WatchStreamAsync()'s IAsyncEnumerable<WorkflowEvent> simply stops producing items. Task.WhenAny against a timeout on the outer pump shows no further activity — no exception is thrown, no cancellation occurs, the enumerator just never completes and never yields again.

Workaround

We cannot prevent this from our side — it is entirely inside MAF's stream/dispatch internals. Our workaround is disclosure + external detection: our wrapper now (1) records every observed event durably as it streams, so a stall leaves a full trace instead of an in-memory-only history that a crash or timeout loses, and (2) runs a separate-process watchdog that polls the durable trace's timestamps and, if a state goes silent for longer than an adaptive (EWMA-based) threshold, first asks the run process to cooperatively stop, then hard-kills it — so a silent MAF stream stall now surfaces as a bounded, diagnosable, named failure instead of consuming an arbitrarily long outer CI/orchestration timeout with an unexplained hang.

Ask

Any of the following would help:

• A known cause / fix already in a later Microsoft.Agents.AI.Workflows version we should upgrade to.
• Guidance on what internal state to inspect (or what to instrument) to distinguish "stream legitimately ended silently because the run is done" from "stream is stalled" — we tried StreamingRun.GetStatusAsync() and found it does not reliably separate the two cases across MAF's fresh-run vs. checkpoint-resume construction paths (RunStatus.Idle was observed on a resumed run that we independently verified reached genuine completion via a WorkflowOutputEvent immediately beforehand).
• Whether there's an existing timeout/heartbeat knob on the workflow dispatch loop we're missing that would surface this as an explicit failure rather than silence.

Environment

• Microsoft.Agents.AI.Workflows / Microsoft.Agents.AI.Workflows.Generators: 1.13.0
• .NET 10, macOS (Apple Silicon) and Linux CI runners
• Execution mode: in-process (InProcessExecution), no distributed/durable-task runtime involved
• Chat client: OpenAI-compatible endpoint (local oMLX server), streaming disabled at the chat-client layer