temporalio / temporalio/temporal
Optimize large payload fan-out to activities and child workflows
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 23.2k
- Forks
- 1.9k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 228
Description
Is your feature request related to a problem? Please describe.
Some use cases need to pass the same payload to a large number of activities or child worklfows. In the current model, the payload will be repeated in the history as many times as the number of invocations. Recording the same payload many times is clearly nonoptimal.
Describe the solution you'd like
I don't have a specific API/feature proposal to resolve this issue. But I believe some solution that saves such a payload only once should be possible.
One simplistic strawman is to have a special command to save a payload and then allow referencing it in other places of a workflow.
Another idea is to perform payload deduplication by storing large payload hashes in the mutable state.
Additional context
A forum post that describes a customer having issues with a large payload passed to many child workflows.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the issue and the linked forum post about workflow history size limits. No implementation files, tests, entry points, or concrete API are identified; the work is not ready to pick up until a design is agreed for storing a large payload once and referencing or deduplicating it across activity and child-workflow invocations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100