V2 of waitForStateCompletion API using synchronous update feature
- Dominant language
- Go
- Stars
- 661
- Forks
- 63
- PR merge metrics
- No merged PRs in 30d
Description
The current implementation uses a sub workflow: https://github.com/indeedeng/iwf/pull/345 requires to provide `WaitingForCompletionStateExecutionIds` on starting workflow which is cumbersome to use. It's required because we want minimize the initiation of these sub workflows as optimization.
Additionally, it's not efficient because of the overhead of starting/signaling another workflow. It consume quite a lot of actions
A better way is:
* Extend the stateExecutionCounter to track the completed state executionId
* Use synchronous update with `workflow.Await( ()-> stateExecutionCounter.stateExecuctionCompleted(stateExeId) )` in the handler to wait for the execution
Note that continueAsNew need to be handled properly:
* In validator, it should check if the workflow has met continueAsNew threshold. If so, then return error
* In the `workflow.Await(...)` should also check continueAsNew threshold to return error so that workflow can do a continueAsNew
* The API service need to retry when continueAsNew interrupts the waiting
* returning state completion output will not be supported
Also, it doesn't allow reuse workflowId for different executions, because of the RunID issues in Temporal. This v2 can solve the problems
~~Note that sync update feature is not prod ready in Temporal yet. We will start working on it when it is ready.~~
Contributor guide
Research direction
Start by reviewing the current waitForStateCompletion implementation and the related sub-workflow approach in linked pull request #345, then trace stateExecutionCounter and the synchronous update entry point. Done means the v2 API handles continue-as-new interruption and retries, avoids the sub-workflow and RunID limitations, and does not return state completion output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend-api-design, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100