microsoft / microsoft/agent-framework

.NET: [Feature] Opt-in asynchronous (dataflow) execution mode — per-edge firing without the superstep barrier

Open
#7,555 0 comments 1 reaction 1 assignee View on GitHub

@peibekwe is already working on this.

Since Aug 10, 2026.

.NET workflows
Dominant language
Python
Stars
13.6k
Forks
2.3k
Avg merge
2d 45m
Merged PRs (30d)
358

Description

## Summary

Request an **opt-in asynchronous (dataflow) execution mode** in which a downstream executor fires as soon as its direct predecessor completes — without waiting for the current superstep to quiesce. BSP (the current model) would remain the default.

## Background

The .NET workflow runtime executes as **Bulk Synchronous Parallel (BSP)**: executors fire in supersteps, and the next superstep begins only after the current one fully quiesces (per the [execution-model docs](https://learn.microsoft.com/en-us/agent-framework/user-guide/workflows/core-concepts/workflows#execution-model) and #1444). This is a sound default — it gives determinism, a simple concurrency model, and clean checkpoint boundaries.

However, as called out in #3840 (and experienced here), BSP has a latency cost in one specific but common graph shape, and there is currently no way to opt out of it.

## The gap

When a fan-out has siblings of **unequal latency**, a downstream chain of the *fast* sibling waits for the *slow* sibling, even though it has no data dependency on it:

```
A ──fan-out──→ B (fast) ──→ D ← D depends only on B
└──→ C (slow)
```

- B finishes quickly, but D is one superstep deeper than B.
- D's superstep cannot start until the current superstep quiesces — i.e. until **C** finishes.
- So D is gated by C, despite D not depending on C.

`WithIntermediateOutputFrom` lets a *completed* executor stream its output early within a superstep, but it cannot make a *not-yet-scheduled* executor start sooner. Sub-workflows do not help either (see #3840): their inner supersteps share the parent scheduler and are subject to the same barrier.

## Minimal repro

Synthetic workflow where "C" is gated by a `TaskCompletionSource` to make the barrier visible (executor bodies omitted for brevity — B and D return instantly, C awaits the gate):

```csharp
// A → fan-out[B, C]; B → D
var a = new SourceExecutor(); // emits a value, then completes
var b = new FastExecutor("B");
var c = new GatedExecutor("C", gate); // blocks until gate.SetResult()
var d = new FastExecutor("D");

var wf = new WorkflowBuilder(a)
.AddFanOutEdge(a, [b, c])
.AddEdge(b, d)
.WithOutputFrom(d)
.Build();

await using var run = await InProcessExecution.RunStreamingAsync(wf, input);
await run.TrySendMessageAsync(new TurnToken(true));
```

Measured (`InProcessExecution` / OffThread, .NET 10, `Microsoft.Agents.AI.Workflows` 1.15.0):

| Executor | Completes at | Notes |
|---|---|---|
| B | ~10 ms | fast; its output (D's only input) is ready |
| C | ~10000 ms | gated, released manually |
| **D** | **~10000 ms** | depends on B only — expected ~10 ms, actually waits for C |

D starts only after C finishes, because D lives in the superstep *after* B, and that superstep is gated by C. The same effect occurs whether the B→D chain is expressed as an edge or wrapped as a sub-workflow (`BindAsExecutor`).

## Proposal

An **opt-in** mode where edges fire **eagerly** — a downstream executor is scheduled as soon as its direct predecessor emits, independent of parallel siblings:

- Default remains **BSP**, preserving determinism, checkpoint semantics, and existing tests.
- An opt-in "asynchronous / dataflow" mode (per-workflow, e.g. on `WorkflowBuilder` or `InProcessExecution`) would trade the global barrier for lower tail latency in graphs where independent chains should not gate each other.
- A lighter alternative that would already cover the common case: a **per-edge "eager" flag**, opting specific edges out of the barrier while leaving the rest of the workflow BSP-synchronized.

## Use case

Progressive-result / streaming pipelines (e.g. multi-arm recommendation or retrieval workflows): several independent arms compute concurrently, and each arm should surface to the client the moment it finishes. A fast arm that ends in a short refinement chain (e.g. produce candidates → judge them) currently waits for a slower sibling arm before its refinement step can even start, adding the sibling's latency to the fast arm's time-to-first-result.

## Related

- #3840 — sub-workflow cannot escape the BSP barrier (closed as a limitation of BSP)
- #1444 — "Define Workflow Concurrency Semantics" epic (BSP documented as the execution model)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.