External Payload Storage Foundation

Is your feature request related to a problem? Please describe.

Some clients, workflows, and activities may produce payloads that are too large such that they cause workflow failures and require workflows to be rearchitected to brake down the data into smaller sizes. Sometimes customers have to write abstractions to offload large data into storage systems, pass around references, and manually retrieve that data.

Describe the solution you'd like

The SDK should provide an abstraction to allow customers to opt into offloading payloads to external storage systems when they are large or for other customer-defined determinations.

• Customers can use pre-built storage abstractions (e.g. drivers) for well-known storage systems (e.g. Amazon S3) or define their own that integrates with the external payload storage abstraction.
• External payload storage should remain transparent to workflows and activities. External payload storage may use additional contextual information to better inform how payloads should be externally stored, if at all.
• Offloaded payloads will be represented with claim information in workflow history so that they can be retrieved at a later time or by other systems using the same storage abstractions.

Per-SDK Tickets

• temporalio/sdk-go#2208
• temporalio/sdk-java#2880
• temporalio/sdk-typescript#2063
• temporalio/sdk-python#1342
• .NET -
• Ruby -
• Rust -
• PHP -
• Temporal CLI - N/A

No file, test, or implementation entry point is named. Start by reviewing the completed per-SDK tickets listed in the issue, then compare them with the remaining .NET, Ruby, Rust, and PHP entries. Done requires a concrete foundation and a defined path for the remaining SDK work.