[Durable Execution]: Support heterogeneous result types in Parallel operations
- Dominant language
- C#
- Stars
- 1.7k
- Forks
- 503
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 18
Description
## Describe the feature
Please add support for branches with different result types within a single
Durable Execution `Parallel` operation.
The current .NET API uses one generic result type for the entire batch:
```csharp
Task> ParallelAsync(
IReadOnlyList> branches,
string? name = null,
ParallelConfig? config = null,
CancellationToken cancellationToken = default);
```
Consequently, every branch must return a value assignable to the same `T`.
Independent branches returning unrelated types, such as `InventoryResult` and
`PaymentResult`, must instead use `object`, a common base type, or an artificial
wrapper/envelope.
The Java Durable Execution SDK supports heterogeneous results by making each
branch independently generic:
```java
DurableFuture inventory;
DurableFuture payment;
try (var parallel = context.parallel("process-order")) {
inventory = parallel.branch(
"inventory", String.class, branch -> "reserved");
payment = parallel.branch(
"payment", Integer.class, branch -> 200);
}
String inventoryResult = inventory.get();
Integer paymentResult = payment.get();
```
Each Java branch declares its own result type and returns a typed
`DurableFuture`, while the parent parallel operation returns a non-generic
`ParallelResult` summary.
The branch-oriented API also allows branches to be registered incrementally.
By contrast, .NET's current `ParallelAsync` API accepts a complete
`IReadOnlyList` of branches up front. A caller cannot create a parallel
operation, add or start work as it is discovered, and then explicitly seal and
await the operation.
References:
* [.NET `ParallelAsync` documentation](https://docs.aws.amazon.com/durable-execution/sdk-reference/operations/parallel/#contextparallel)
* [Java `ParallelDurableFuture`](https://github.com/aws/aws-durable-execution-sdk-java/blob/main/sdk/src/main/java/software/amazon/lambda/durable/ParallelDurableFuture.java)
* [Java heterogeneous parallel conformance test](https://github.com/aws/aws-durable-execution-sdk-java/blob/main/conformance-tests/src/main/java/parallel/ParallelHeterogeneous.java)
## Use Case
A parallel operation commonly coordinates independent work whose outputs are
intrinsically different, for example:
* reserve inventory and return `InventoryReservation`
* authorize payment and return `PaymentAuthorization`
* calculate shipping and return `ShippingQuote`
Requiring one batch-level `T` makes this workflow less type-safe or forces
unrelated branch contracts into a shared wrapper.
Incremental registration is also useful for agentic AI workloads. An agent may
checkpoint a generated plan, discover tool calls or specialist tasks from that
plan, and add branches as those tasks become known. Requiring a fully
materialized branch list before starting any branch adds coordination
boilerplate and prevents earlier independent work from starting while later
branches are still being assembled.
Supporting branch-scoped result types would be preferable for several reasons:
1. **End-to-end type safety.** Each result retains its concrete compile-time
type without casts, type switches, or optional fields in a shared envelope.
2. **Reliable checkpoint deserialization.** An explicit type per branch gives
the serializer the correct target type during replay. Using `object` can
deserialize into `JsonElement` or require custom polymorphic configuration.
3. **Better Native AOT support.** Concrete branch result types can be registered
with a source-generated `JsonSerializerContext`; an open-ended `object`
result is difficult to make trimming-safe.
4. **Natural modeling.** Parallel branches are independent operations, so their
public result contracts should not need an inheritance relationship solely
because they execute together.
5. **Less boilerplate.** Users would not need artificial union/envelope types
or manual result discrimination.
6. **Cross-SDK parity and portability.** Workflows can be translated between
Java and .NET without weakening their type contracts.
7. **Separation of concerns.** The parent operation can continue to expose
aggregate completion status and counts, while typed branch handles expose
individual values and errors.
8. **Incremental composition.** Callers can register and start branches as a
workflow plan is built, then explicitly complete the registration phase and
await the aggregate result. This better supports dynamic fan-out and agentic
orchestration than requiring every branch to be predefined in one list.
This appears to be an SDK API limitation rather than a service limitation,
because the Java SDK's conformance suite checkpoints and restores heterogeneous
branch results.
## Proposed Solution
Please consider adding an additive, branch-oriented API alongside the existing
homogeneous `ParallelAsync` overloads. The exact naming is open for
discussion, but an illustrative shape could be:
```csharp
await using var parallel = context.CreateParallel(
name: "process-order",
config: new ParallelConfig());
var inventory = parallel.BranchAsync(
"inventory",
async (branch, ct) => await ReserveInventoryAsync(branch, ct));
var payment = parallel.BranchAsync(
"payment",
async (branch, ct) => await AuthorizePaymentAsync(branch, ct));
if (plan.RequiresComplianceReview)
{
var compliance = parallel.BranchAsync(
"compliance",
async (branch, ct) => await ReviewComplianceAsync(branch, ct));
}
IBatchResult summary = await parallel.CompleteAsync();
InventoryReservation inventoryResult = await inventory;
PaymentAuthorization paymentResult = await payment;
```
This is intended only as an API sketch. Other idiomatic .NET designs would also
work if they provide:
* a result type declared independently for each branch
* a typed handle for retrieving each branch's value or error
* incremental branch registration until the operation is explicitly completed
or sealed, with branches allowed to start as they are registered
* an aggregate parallel completion result
* support for existing `MaxConcurrency`, `CompletionConfig`, `NestingType`,
naming, cancellation, and deterministic replay behavior
* serialization through the registered `ILambdaSerializer`, including
source-generated serializers
Incremental registration must retain Durable Execution's deterministic replay
contract. For example, an AI-generated plan or set of tool calls should be
produced in a checkpointed step, so replay registers the same branches in the
same order. The requested API should support dynamic composition without
weakening replay validation.
The existing `ParallelAsync` API should remain available for homogeneous
batches and convenient `GetResults()` usage.
## Other Information
Current workarounds are:
* use `object`, losing useful static and serialization type information
* define a common base type and configure polymorphic serialization
* return a shared envelope with optional fields for every possible branch
* avoid `ParallelAsync` and manually compose child contexts
None provides the same combination of type safety, replay-safe deserialization,
incremental composition, and ergonomics as Java's branch-scoped generic API.
## Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
## AWS .NET SDK and/or Package version used
`Amazon.Lambda.DurableExecution` 1.0.0
## Targeted .NET Platform
.NET 10
## Operating System and version
N/A (cross-platform API design request)
Contributor guide
Research direction
Start with the existing .NET ParallelAsync API and compare it with Java's ParallelDurableFuture.java and the ParallelHeterogeneous.java conformance test. Define an additive branch-oriented design that preserves typed handles, incremental registration, aggregate completion, serialization, and deterministic replay while retaining homogeneous batches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, csharp, java
- Domain
- api, backend-api-design, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100