aws / aws/aws-durable-execution-sdk-rust

Support strongly typed heterogeneous results in a single Parallel operation

Open
#65 0 comments 0 reactions 0 assignees View on GitHub
enhancement needs-triage pkg:sdk
Dominant language
Rust
Stars
13
Forks
0
Avg merge
10m
Merged PRs (30d)
1

Description

## Problem

The current Parallel API is intentionally homogeneous at the result boundary:

```rust
pub fn parallel(&self, branches: Branches) -> ParallelBuilder
where
Branches: IntoIterator>,
O: Serialize + DeserializeOwned + Send + 'static,
```

Every branch must therefore produce the same `O`, and the operation returns `Vec` or `BatchResult`. The implementation already erases differing closure and future types inside `BranchBody`, but preserves one shared output type for the branch collection.

For branches with unrelated results, callers currently need to:

- erase values to `serde_json::Value`
- define an enum or shared wrapper
- run independently typed durable builders with `tokio::join!`

The existing `tokio::join!` approach preserves result types, but it coordinates separate durable operations rather than one durable Parallel parent. It therefore does not provide the same parent Parallel checkpoint, branch hierarchy, aggregate summary, max-concurrency behavior, or completion policy.

Common workflows naturally have unrelated branch contracts, for example inventory reservation, payment authorization, and shipping quotation.

## Desired behavior

Provide an additive API that supports:

- a concrete result type declared independently for each branch
- typed result retrieval without `Any`, JSON value inspection, or downcasts
- one durable Parallel parent and normal ParallelBranch child checkpoints
- deterministic branch identity and replay
- existing max concurrency, completion policy, nesting, naming, and serialization behavior
- ideally incremental registration, allowing an earlier branch to start or be awaited before registration is sealed

Keep the existing homogeneous `parallel(Vec>)` API for dynamic fan-out where every branch naturally returns the same type.

## Possible designs

### 1. Java-style typed branch handles

```rust
let mut parallel = ctx.create_parallel("process-order");

let inventory: BranchHandle = parallel.branch(
"inventory",
|ctx| async move { reserve_inventory(ctx).await },
);

let payment: BranchHandle = parallel.branch(
"payment",
|ctx| async move { authorize_payment(ctx).await },
);

let summary = parallel.complete().await?;
let inventory = inventory.await?;
let payment = payment.await?;
```

The parent can retain type-erased internal branch controllers while each public `BranchHandle` owns a typed result receiver and branch-specific deserializer. Explicit async completion is preferable to relying on `Drop`, because Rust does not support async drop.

This design also naturally supports incremental registration and early branch start.

Java reference: https://github.com/aws/aws-durable-execution-sdk-java/blob/main/sdk/src/main/java/software/amazon/lambda/durable/ParallelDurableFuture.java

### 2. Tuple-based API or macro

```rust
let (inventory, payment) = ctx
.parallel_tuple((
Branch::new("inventory", |ctx| async move { reserve_inventory(ctx).await }),
Branch::new("payment", |ctx| async move { authorize_payment(ctx).await }),
))
.await?;
```

Alternatively, a `parallel!` macro could return a typed tuple. This is ergonomic for a fixed compile-time branch set and avoids public type erasure. Since Rust has no variadic generics, it would likely require tuple implementations for supported arities or macro expansion.

The tuple API could be sugar over the typed-handle implementation so both forms share checkpoint and replay machinery.

## Compatibility and serialization

This should be additive. Each branch must serialize and deserialize using its own concrete result type while the parent still records a type-erased batch summary. Replay must restore each typed handle without rerunning completed branch code.

For runtime-sized heterogeneous collections, an explicit enum remains the appropriate Rust model; this request focuses on fixed or incrementally registered branches whose individual types are known to the caller.

## Conformance context

Two optional cross-SDK requirements are being proposed for this capability:

- independently typed heterogeneous branch handles
- branch execution before registration is sealed

Conformance issue: https://github.com/aws/aws-durable-execution-conformance-tests/issues/74
Conformance PR: https://github.com/aws/aws-durable-execution-conformance-tests/pull/112

## Acceptance criteria

- [ ] Select and document a public API design, potentially supporting both handle and tuple ergonomics.
- [ ] Demonstrate at least two branches with unrelated concrete result types.
- [ ] Preserve a single Parallel parent and normal ParallelBranch checkpoint hierarchy.
- [ ] Verify initial execution and replay restore typed results without rerunning completed branches.
- [ ] Cover branch-specific serialization and deserialization.
- [ ] Document when to use homogeneous Parallel, a typed heterogeneous form, or an enum.

Contributor guide

Open the contributing guide

Research direction

Start by reading the current Parallel API, BranchBody implementation, and existing homogeneous parallel behavior. Compare the proposed handle and tuple designs against checkpoint hierarchy, replay, and branch-specific serialization requirements. Done means a selected documented API, typed heterogeneous results, preserved parent and child checkpoints, replay coverage, and guidance on homogeneous, heterogeneous, and enum use.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.