temporalio / temporalio/temporal
Support data references to avoid storing the same Payload multiple times
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 23.2k
- Forks
- 1.9k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 228
Description
Is your feature request related to a problem? Please describe.
The same value can be an argument of many activities. For example, one activity returns a document, and then 1k activities are invoked, each of them receiving a document as one of the arguments. In this case, the document is going to be stored 1k+1 times in the history which is wasteful.
Describe the solution you'd like
My strawman is to introduce a special command (or reuse Marker command) that would store a value (as Payloads) in the history. Add support for activity/child workflow inputs and outputs reference that value (by eventId) instead of including it directly into the corresponding schedule commands.
Another option is to support referencing Payloads directly inside other events like WorkflowExecutionStarted.
Additional context
From Slack:
To make this more concrete, i'm implementing a DSL on top of a Temporal workflow.
The DSL workflow definition has some things that need to be extracted and consumed throughout, sometimes.
Currently i'm passing around my DSL string to all functions as a "dsl workflow context"
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by studying the proposed Marker command, Payload storage, activity and child workflow input/output references, schedule commands, and WorkflowExecutionStarted events described in the issue. Define the event and history semantics before implementation; done would include a decided reference model that avoids duplicate Payload storage while preserving workflow behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100