S3 External Storage Driver

Is your feature request related to a problem? Please describe.

The external payload storage foundation described in https://github.com/temporalio/features/issues/765 enables payloads to be externally stored for various customer reasons. But there are no pre-built storage drivers for customers to use, thus requiring customers to write their own drivers. The SDK should provide ones with sufficient demand, such as an AWS S3 driver.

Describe the solution you'd like

A pre-built storage driver that can handle most customers needs for storing their large payloads in AWS S3.

• Can store and retrieve from buckets that are statically named or dynamically determined based on payload information and execution context.
• Stores payloads with object keys in a hierarchy that allows customers to implement lifecycle management easily (e.g. delete all keys that correspond to run X of workflow Y).
• Stores payloads with object keys to maximize the deduplication of storing the same payload data within certain context levels. For example, a large payload that is passed in and out via activity inputs and results within the same workflow is ideally deduplicated to a single object key to optimize storage and prevent unnecessary transfers.
• Driver is a separate module/package/bundle that doesn't pull in S3 dependencies unless actually required based on inclusion of the driver.

Per-SDK Tickets

• temporalio/sdk-go#2251
• temporalio/sdk-java#2882
• temporalio/sdk-typescript#2065
• temporalio/sdk-python#1390
• .NET -
• Ruby -
• Rust -
• PHP -
• Temporal CLI -

Start by reading features issue 765 and the checked and unchecked per-SDK tickets listed here to determine the applicable SDK scope. Define the driver against the stated S3 requirements: dynamic or static buckets, hierarchical and deduplicated object keys, and optional S3 dependencies. Done means the selected SDK has a usable separate driver package meeting those requirements.