PipedreamHQ / PipedreamHQ/pipedream
[FEATURE] Step Memoisation
- Dominant language
- JavaScript
- Stars
- 11.7k
- Forks
- 5.8k
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 102
Description
**Is your feature request related to a problem? Please describe.**
As we break down logic into individual steps, there are times we need to interact with another system in a non-idempotent way (side affect / mutation). Rerunning old events to test code fixes can become cumbersome as "skipping" steps isn't trivial, especially when they output something needed further down the chain.
**Describe the solution you'd like**
The ability to mark a step to memoise. The inputs for the step should be calculate based on the code, not the entire event chain.
IE, the memoisation key would be a composite key of the values of any `event.`, `param.`, or `step.` in the steps code.
When the calculated inputs have been seen before, the exports and return value can be replayed.
**Do you have a workaround?**
Conditional checkpoint logic
Contributor guide
Assessment
This issue has not been assessed yet.