aws / aws/aws-lambda-dotnet

[Durable Execution]: Support heterogeneous result types in Parallel operations

Offen
#2,519 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
feature-request needs-review
Vorherrschende Sprache
C#
Sterne
1.7k
Forks
503
Ø Merge
1 T. 18 Std.
Gemergte PRs (30 T.)
21

Beschreibung

## Describe the feature

Please add support for branches with different result types within a single
Durable Execution `Parallel` operation.

The current .NET API uses one generic result type for the entire batch:

```csharp
Task> ParallelAsync(
IReadOnlyList> branches,
string? name = null,
ParallelConfig? config = null,
CancellationToken cancellationToken = default);
```

Consequently, every branch must return a value assignable to the same `T`.
Independent branches returning unrelated types, such as `InventoryResult` and
`PaymentResult`, must instead use `object`, a common base type, or an artificial
wrapper/envelope.

The Java Durable Execution SDK supports heterogeneous results by making each
branch independently generic:

```java
DurableFuture inventory;
DurableFuture payment;

try (var parallel = context.parallel("process-order")) {
inventory = parallel.branch(
"inventory", String.class, branch -> "reserved");
payment = parallel.branch(
"payment", Integer.class, branch -> 200);
}

String inventoryResult = inventory.get();
Integer paymentResult = payment.get();
```

Each Java branch declares its own result type and returns a typed
`DurableFuture`, while the parent parallel operation returns a non-generic
`ParallelResult` summary.

The branch-oriented API also allows branches to be registered incrementally.
By contrast, .NET's current `ParallelAsync` API accepts a complete
`IReadOnlyList` of branches up front. A caller cannot create a parallel
operation, add or start work as it is discovered, and then explicitly seal and
await the operation.

References:

* [.NET `ParallelAsync` documentation](https://docs.aws.amazon.com/durable-execution/sdk-reference/operations/parallel/#contextparallel)
* [Java `ParallelDurableFuture`](https://github.com/aws/aws-durable-execution-sdk-java/blob/main/sdk/src/main/java/software/amazon/lambda/durable/ParallelDurableFuture.java)
* [Java heterogeneous parallel conformance test](https://github.com/aws/aws-durable-execution-sdk-java/blob/main/conformance-tests/src/main/java/parallel/ParallelHeterogeneous.java)

## Use Case

A parallel operation commonly coordinates independent work whose outputs are
intrinsically different, for example:

* reserve inventory and return `InventoryReservation`
* authorize payment and return `PaymentAuthorization`
* calculate shipping and return `ShippingQuote`

Requiring one batch-level `T` makes this workflow less type-safe or forces
unrelated branch contracts into a shared wrapper.

Incremental registration is also useful for agentic AI workloads. An agent may
checkpoint a generated plan, discover tool calls or specialist tasks from that
plan, and add branches as those tasks become known. Requiring a fully
materialized branch list before starting any branch adds coordination
boilerplate and prevents earlier independent work from starting while later
branches are still being assembled.

Supporting branch-scoped result types would be preferable for several reasons:

1. **End-to-end type safety.** Each result retains its concrete compile-time
type without casts, type switches, or optional fields in a shared envelope.
2. **Reliable checkpoint deserialization.** An explicit type per branch gives
the serializer the correct target type during replay. Using `object` can
deserialize into `JsonElement` or require custom polymorphic configuration.
3. **Better Native AOT support.** Concrete branch result types can be registered
with a source-generated `JsonSerializerContext`; an open-ended `object`
result is difficult to make trimming-safe.
4. **Natural modeling.** Parallel branches are independent operations, so their
public result contracts should not need an inheritance relationship solely
because they execute together.
5. **Less boilerplate.** Users would not need artificial union/envelope types
or manual result discrimination.
6. **Cross-SDK parity and portability.** Workflows can be translated between
Java and .NET without weakening their type contracts.
7. **Separation of concerns.** The parent operation can continue to expose
aggregate completion status and counts, while typed branch handles expose
individual values and errors.
8. **Incremental composition.** Callers can register and start branches as a
workflow plan is built, then explicitly complete the registration phase and
await the aggregate result. This better supports dynamic fan-out and agentic
orchestration than requiring every branch to be predefined in one list.

This appears to be an SDK API limitation rather than a service limitation,
because the Java SDK's conformance suite checkpoints and restores heterogeneous
branch results.

## Proposed Solution

Please consider adding an additive, branch-oriented API alongside the existing
homogeneous `ParallelAsync` overloads. The exact naming is open for
discussion, but an illustrative shape could be:

```csharp
await using var parallel = context.CreateParallel(
name: "process-order",
config: new ParallelConfig());

var inventory = parallel.BranchAsync(
"inventory",
async (branch, ct) => await ReserveInventoryAsync(branch, ct));

var payment = parallel.BranchAsync(
"payment",
async (branch, ct) => await AuthorizePaymentAsync(branch, ct));

if (plan.RequiresComplianceReview)
{
var compliance = parallel.BranchAsync(
"compliance",
async (branch, ct) => await ReviewComplianceAsync(branch, ct));
}

IBatchResult summary = await parallel.CompleteAsync();

InventoryReservation inventoryResult = await inventory;
PaymentAuthorization paymentResult = await payment;
```

This is intended only as an API sketch. Other idiomatic .NET designs would also
work if they provide:

* a result type declared independently for each branch
* a typed handle for retrieving each branch's value or error
* incremental branch registration until the operation is explicitly completed
or sealed, with branches allowed to start as they are registered
* an aggregate parallel completion result
* support for existing `MaxConcurrency`, `CompletionConfig`, `NestingType`,
naming, cancellation, and deterministic replay behavior
* serialization through the registered `ILambdaSerializer`, including
source-generated serializers

Incremental registration must retain Durable Execution's deterministic replay
contract. For example, an AI-generated plan or set of tool calls should be
produced in a checkpointed step, so replay registers the same branches in the
same order. The requested API should support dynamic composition without
weakening replay validation.

The existing `ParallelAsync` API should remain available for homogeneous
batches and convenient `GetResults()` usage.

## Other Information

Current workarounds are:

* use `object`, losing useful static and serialization type information
* define a common base type and configure polymorphic serialization
* return a shared envelope with optional fields for every possible branch
* avoid `ParallelAsync` and manually compose child contexts

None provides the same combination of type safety, replay-safe deserialization,
incremental composition, and ergonomics as Java's branch-scoped generic API.

## Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

## AWS .NET SDK and/or Package version used

`Amazon.Lambda.DurableExecution` 1.0.0

## Targeted .NET Platform

.NET 10

## Operating System and version

N/A (cross-platform API design request)

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne mit der bestehenden .NET ParallelAsync API und vergleiche sie mit Java's ParallelDurableFuture.java und dem Conformance-Test ParallelHeterogeneous.java. Definiere ein additives, zweigorientiertes Design, das typisierte Handles, inkrementelle Registrierung, aggregierte Fertigstellung, Serialisierung und deterministisches Replay bewahrt und gleichzeitig homogene Batches beibehält.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
aws, csharp, java
Bereich
api, backend-api-design, distributed-systems
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.