microsoft / microsoft/agent-framework-durable-extension
Let each workflow carry its own version instead of one DefaultVersion for the whole graph
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 16
- Forks
- 10
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 9
Description
Description
A sub-workflow cannot carry its own version. Every sub-orchestration in a workflow graph falls back to
the worker's single Versioning.DefaultVersion, because the extension never supplies one; the root's
version comes from the client at schedule time. So there is exactly one version stamp per worker for the
whole graph, and no way to distinguish the workflow that changed from the ones that didn't.
A topology change to one leaf workflow therefore invalidates every in-flight run of the graph. For an
application that has opted into worker versioning, the generated-orchestrator model pushes it to
MatchStrategy.Strict (see below), and the previous worker deployment must then stay alive until the
whole population drains. Our runs park on RequestPort gates
waiting for humans, for days or weeks, so a one-line change to a leaf workflow pins an old worker
deployment in every environment for as long as the longest outstanding human wait.
The execution model already has the isolation — each sub-workflow runs as its own sub-orchestration.
Only the version stamp is shared.
The runtime already supports this
Every piece of this exists in Microsoft.DurableTask; none of it is reachable from the extension.
- Per-sub-orchestration versions —
SubOrchestrationOptions.Version, present since at least 1.18.0.
A sub-orchestration's version is independent of its parent's, falling back to the worker's
DefaultVersiononly when the caller passes nothing (v1.25.0TaskOrchestrationContextWrapper):string version = options?.Version is { } v ? v.Version : this.GetDefaultVersion(); - Several versions of one orchestration name in one worker —
AddOrchestratorFunc(name, version, factory),
added in 1.25.0 (durabletask-dotnet#695).
Dispatch routes each work item to its matching version, falling back to an unversioned registration
when the name has none.UseWorkItemFilters()follows suit — per #695 it emits the concrete distinct
version set actually registered for each name (Strictoverrides it with the single worker version) —
so the filters cooperate with multi-version registration rather than fighting it.
Together these take the drain off the deployment path: keep the old and new implementations of the changed
workflow registered side by side under CurrentOrOlder, and every in-flight instance keeps replaying
against the code it started on. No second worker deployment, and nothing to wait for before shipping. (The
old graph does have to stay registered until its instances drain — removing it early yields
OrchestratorTaskNotFound under #695's dispatch rule — but that is a later cleanup step, not a gate on
the deploy.)
Activities need the same treatment, and this is the easy part to miss. They inherit the parent
instance's version, since they execute in the parent's history — so two versions of one workflow dispatch
the same activity name at two different versions. With today's name-only activity registration, #695's
fallback rule resolves both to the single unversioned implementation, and a 1.2.0 child would execute
1.3.0's executor code: exactly the nondeterminism this issue is trying to prevent. #695 provides
AddActivityFunc(name, version, factory) for precisely this case.
Where the extension closes the door
Two call sites, both on main @ ad941ef:
1. DurableExecutorDispatcher.cs#L203
— the two-argument overload, so no version can be supplied:
DurableWorkflowResult? workflowResult = await context.CallSubOrchestratorAsync<DurableWorkflowResult?>(
orchestrationName,
workflowInput).ConfigureAwait(true);
2. ServiceCollectionExtensions.cs#L255
— name-only registration, so the multi-version registry is never engaged:
registry.AddOrchestratorFunc<DurableWorkflowInput<object>, DurableWorkflowResult>(
registration.OrchestrationName,
(context, input) => RunWorkflowOrchestrationAsync(context, input, durableOptions));
Applications can't work around it either: AddWorkflow takes no version, and DurableWorkflowRunner is
internal sealed with an internal RunWorkflowOrchestrationAsync.
Why we're on Strict, and why this ask doesn't depend on it
Worth separating two things the word "versioning" covers: per-task registration versioning (which
implementation runs for a given (name, version)) and worker admission policy (UseVersioning's
MatchStrategy, which instances a worker will accept at all). This issue is about the first.
They really are independent — MatchStrategy defaults to VersionMatchStrategy.None
(DurableTaskWorkerOptions.cs), and the SDK's own ActivityVersioningSample at v1.25.0 registers
versioned orchestrators and activities and schedules across them without calling UseVersioning at
all. So per-workflow versions would be useful to the extension regardless of what admission policy an
application picks; nothing here forces one on anyone.
For context on why we ended up at Strict + Reject specifically: CurrentOrOlder assumes the
orchestrator author can write a context.Version branch to keep an older instance's replay
deterministic. Here the orchestrator body is generated from the graph, so there is nowhere to put that
branch, and a newer worker accepting an older instance faults it on replay. Strict avoids the fault;
the price is that the version stamp is all-or-nothing across the graph. Per-workflow versions remove the
need for the branch entirely — each version gets its own registered graph — which is what would make
CurrentOrOlder viable for a generated-orchestrator model.
Proposal
Scope: standalone workers. The Azure Functions hosting surface can't take this as-is and is called out
separately below.
Prerequisite: the extension pins Microsoft.DurableTask.* to 1.18.0
(dotnet/Directory.Packages.props), and the version-aware AddOrchestratorFunc(name, version, factory) /
AddActivityFunc(name, version, factory) overloads only exist from 1.25.0. So none of the below
compiles until that pin moves; the bump and its compatibility validation are part of the work, not a
precondition someone else handles. (For what it's worth, we moved our own app from 1.18.0 to 1.25.0 with
no code changes and no test failures.)
- accept an optional version with the workflow —
AddWorkflow(workflow, version), or aVersionproperty
onWorkflowif that composes better with the declarative and Python surfaces; - register orchestrations via
AddOrchestratorFunc(name, version, factory), and their activities via
AddActivityFunc(name, version, factory), when a version is present; - schedule the child at its registered version from
ExecuteSubWorkflowAsync(see the sample below); - carry the version through every name-keyed structure in registration, not just the one. At
ad941ef
there are four, and missing any single one silently drops the second version rather than failing loudly:DurableWorkflowOptions._workflows, keyed onworkflow.Name;BuildWorkflowRegistrationRecursive'sregisteredOrchestrations—
if (!registeredOrchestrations.Add(orchestrationName)) { return; }skips a duplicate name by early
return, so the second version is never registered and its sub-workflow subtree is never walked;BuildWorkflowRegistration'sregisteredActivities(aHashSet<string>shared across the whole
graph walk) together with the name-onlyAddActivityFuncabove;ExecutorRegistry.Register—_executors.TryAdd(executorName, ...), first-writer-wins and never
evicted. This is the registry #50 identifies as the source of "B's topology while executing A's code";
two versions of one workflow share executor names and would hit it by construction.
- version the top-level start paths too, not just child dispatch.
DurableWorkflowClient.RunAsyncand
StreamAsynccallScheduleNewOrchestrationInstanceAsyncwithStartOrchestrationOptionscarrying only
an instance id, so a root's version comes solely from the client's singleUseDefaultVersion. Once
top-level workflows carry their own versions that default is wrong for all but one of them — schedule
workflow A (v1.1) while the client default is v1.4 and exact(name, version)dispatch finds no
implementation. These surfaces should resolve and pass the registered version, and the design needs an
answer for what a name-only start does when a name has several versions (newest wins? require an
explicit version? configurable?).
Backward compatible: an unversioned workflow behaves exactly as today, which is also #695's
unversioned-fallback rule.
Azure Functions hosting needs its own answer
Same-name multi-version registration is explicitly unsupported in Azure Functions per #695, which emits
DURABLE3004 for it — function names derive from the durable task name, so two same-named tasks generate
colliding triggers. The extension's own DurableWorkflowsFunctionMetadataTransformer has exactly that
shape: it walks workflowOptions.Workflows, derives WorkflowNamingHelper.ToOrchestrationFunctionName(workflow.Key)
and the workflows/{workflow.Key}/run HTTP route from the workflow name alone, and dedupes through a
registeredFunctions HashSet<string>. Two versions of one workflow would collapse to a single set of
triggers.
So this proposal is scoped to standalone workers. The Functions surface needs either its own design or an
explicit, early failure — rejecting a versioned workflow under Functions hosting with a message pointing
at DURABLE3004 would be far better than the silent collapse the transformer would produce today.
Worth settling against #50 / PR #66 first
Open PR #66 makes AddWorkflow
throw on a duplicate name, fixing #50.
Right fix — but it hardens name uniqueness, the invariant this proposal needs widened, since hosting v1
and v2 of a changed workflow means two registrations sharing a name.
Keying on (name, version) reconciles both: duplicate unversioned names still throw (#50 stays fixed),
distinct versions under one name become legal, and a true (name, version) collision still throws. The
same widening applies to the other three name-keyed structures listed above, ExecutorRegistry in
particular — it is the one #50 actually blames, and a name-only key there defeats the purpose even if
AddWorkflow is fixed. Cheap to align while #66 is open; a breaking change afterwards.
Narrower fallback
Exposing a hook to supply SubOrchestrationOptions on sub-workflow dispatch would, on its own, let
applications version their children independently — leaving them to manage registration by hand.
Alternatives considered
CurrentOrOlderwith a single version — unsafe, per above.- Application-side versioned registration — blocked by
DurableWorkflowRunnerbeinginternal. ContinueAsNewOptions.NewVersion(durabletask-dotnet 1.23.1) — not exposed by the extension either,
and wouldn't preserve an in-flight run's position mid-graph.- Never changing a workflow's topology while runs are parked — what we do today; it makes deployment
cadence a function of human response time.
Code sample
// Only the workflow that actually changed gets a new version.
// Both implementations stay registered, so nothing needs to drain.
services.ConfigureDurableWorkflows(
workflows =>
{
workflows.AddWorkflow(rootWorkflow, version: "1.4.0"); // unchanged
workflows.AddWorkflow(decompositionWorkflow, version: "1.1.0"); // unchanged
workflows.AddWorkflow(framingWorkflowV1, version: "1.2.0"); // retained for in-flight
workflows.AddWorkflow(framingWorkflowV2, version: "1.3.0"); // new topology
},
workerBuilder: worker =>
{
worker.UseDurableTaskScheduler(connectionString, credential);
// UseVersioning is optional — per-task registration versioning works without it
// (MatchStrategy defaults to None). This is our admission posture, not a requirement
// of the feature.
worker.UseVersioning(new DurableTaskWorkerOptions.VersioningOptions
{
// Must be >= the highest per-workflow version above, or CurrentOrOlder
// rejects those instances and they re-queue forever.
Version = "1.4.0",
DefaultVersion = "1.4.0",
MatchStrategy = DurableTaskWorkerOptions.VersionMatchStrategy.CurrentOrOlder,
FailureStrategy = DurableTaskWorkerOptions.VersionFailureStrategy.Reject,
});
});
// ExecuteSubWorkflowAsync would resolve the child's registered version and schedule with it:
//
// await context.CallSubOrchestratorAsync<DurableWorkflowResult?>(
// orchestrationName,
// workflowInput,
// childVersion is null ? null : new TaskOptions { Version = new TaskVersion(childVersion) });
//
// (TaskOptions.Version, not SubOrchestrationOptions.Version — the latter is a shadowing property
// kept for binary compatibility, and 1.25.0's own docs point new code at the base.)
//
// A run parked on a gate under root 1.4.0 resumes on the new worker. A framing child already in
// flight at 1.2.0 keeps replaying against framingWorkflowV1. New framing children get 1.3.0.
Acceptance criteria
Microsoft.DurableTask.*moved from 1.18.0 to 1.25.0 indotnet/Directory.Packages.props, solution
building and existing tests passing against it.AddWorkflowaccepts an optional version, and registration keys on(name, version)rather than name
across all four structures listed above —_workflows,registeredOrchestrations,
registeredActivities, andExecutorRegistry.- Orchestrations and activities registered through the version-aware
AddOrchestratorFunc/
AddActivityFuncoverloads when a version is present; unversioned workflows register exactly as today. ExecuteSubWorkflowAsyncschedules each child at its registered version.DurableWorkflowClient.RunAsync/StreamAsyncresolve and pass the registered version, with a
documented, tested answer for what a name-only start does when a name has several versions.- Azure Functions hosting either gains a working design or rejects a versioned workflow at startup with an
explicit error referencingDURABLE3004— not the silent trigger collapse the metadata transformer
would produce today. - Tests covering two versions of one workflow co-hosted in a single worker: an in-flight v1 instance keeps
replaying v1's graph and v1's executors, while newly started instances get v2.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with dotnet/Directory.Packages.props and the registration paths in ServiceCollectionExtensions.cs, DurableExecutorDispatcher.cs, DurableWorkflowOptions, and ExecutorRegistry.Register. Trace BuildWorkflowRegistrationRecursive, BuildWorkflowRegistration, ExecuteSubWorkflowAsync, DurableWorkflowClient.RunAsync, and StreamAsync, then review PR #66 and issue #50. Done means standalone workers can register and dispatch distinct workflow and activity versions without breaking unversioned workflows; Azure Functions behavior must be explicitly handled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100