aws / aws/aws-durable-execution-sdk-js

[Feature]: Add composable async SerDes transformation pipelines

Open
#861 0 comments 0 reactions 0 assignees View on GitHub
enhancement parity pkg:sdk
Dominant language
TypeScript
Stars
84
Forks
28
Avg merge
1d 19h
Merged PRs (30d)
43

Description

## What would you like?

Add a composable asynchronous SerDes processing pipeline while preserving the existing `Serdes` interface as the value-codec boundary.

Today `createFileSystemSerdes()` is a complete root SerDes and directly uses `JSON.stringify`/`JSON.parse`. Combining it with a custom class/value SerDes, compression, encryption, signing, or another reversible transformation requires a bespoke wrapper.

The proposed model is:

```text
value -> root Serdes -> string stage -> string stage -> checkpoint
value <- root Serdes <- string stage <- string stage <- checkpoint
```

## Possible Implementation

- Introduce an async `SerdesStage` interface whose serialize/deserialize methods accept and return `string | undefined` plus `SerdesContext`.
- Add a composition factory that starts with one existing `Serdes`, applies stages in forward order during serialization, and reverses them during deserialization.
- Require stages to emit a self-identifying/versioned format. During deserialization, a stage should:
- reverse valid input in its own format;
- throw for malformed input that identifies itself as that stage;
- return unrecognized input unchanged.
- Keep existing `Serdes`, `configureSerdes()`, and operation configuration APIs backward compatible.
- Expose filesystem storage as a stage, or let `createFileSystemSerdes()` accept a configurable root value SerDes, so it is no longer limited to direct JSON serialization.
- Add tests for async stage order, pass-through of external invocation input, `undefined`, malformed recognized envelopes, custom class SerDes, and stages before/after filesystem storage.

## Is this a breaking change?

No.

## Does this require an RFC?

Yes.

## Additional Context

This would preserve TypeScript's native async SerDes advantage while making transformations reusable and independently composable. A comparable Java pipeline design is being developed in aws/aws-durable-execution-sdk-java#648.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing Serdes, configureSerdes(), and createFileSystemSerdes() implementations and operation-configuration APIs. Before coding, turn the proposed model into the required RFC, resolving stage format/versioning, filesystem placement, and backward-compatibility boundaries. Done means an accepted RFC followed by implementation and tests covering the listed async ordering, pass-through, undefined, malformed-envelope, custom-class, and filesystem-order cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.