apple / apple/swift-temporal-sdk
[Feature Discussion] External Storage
- Dominant language
- Swift
- Stars
- 315
- Forks
- 31
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 4
Description
Hey folks,
I'm on the Temporal SDK team and I'm currently implementing [External Storage](https://docs.temporal.io/external-storage) in the Temporal [TypeScript and Java SDKs](https://github.com/temporalio/sdk-typescript/pull/2092) and I'd like to gauge interest in bringing it to the Swift SDK. I'd be happy to put together a high-level design for review that aligns with the other SDKs. I would also have some time over the next few months to contribute PRs.
For context, External Storage offloads large payloads to providers like S3, GCS, or Azure so they don't take up space in your workflow history. Instead of storing the large payload, the history only contains a small claim check. Workers use these claim checks to retrieve the payloads on-demand. As for level of effort, there would be roughly 6-7 units of work for an initial release:
- Pull in new proto types (reference payload) and create base extstore types.
- Implement a payload visitor. Need a way to walk proto messages to find payloads.
- DataConverter wiring. run extstore after codec on encode, before codec on decode.
- StorageDriver interface and built-in drivers. store/retrieve interface, content-addressed keys, hashing, integrity check, cancellation, etc
- Reference payload caching. cache previously stored/retrieved references
- Config and metrics. size thresholds, drivers selector, driver registry
For reference, [here is the Python External Storage feature](https://docs.temporal.io/develop/python/data-handling/external-storage) in Public Preview.
Contributor guide
Research direction
Start by reading the linked Temporal External Storage documentation and comparing the TypeScript/Java PR 2092 and Python feature. Then review the six to seven proposed work areas: proto types, payload visiting, DataConverter wiring, storage drivers, caching, configuration, and metrics. Done means an agreed design and an initial Swift SDK release covering the selected scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100