indeedeng / indeedeng/iwf

V2 of waitForStateCompletion API using synchronous update feature

Open
#349 1 comment 0 reactions 0 assignees View on GitHub
in-review large
Dominant language
Go
Stars
661
Forks
63
PR merge metrics
No merged PRs in 30d

Description

The current implementation uses a sub workflow: https://github.com/indeedeng/iwf/pull/345 requires to provide `WaitingForCompletionStateExecutionIds` on starting workflow which is cumbersome to use. It's required because we want minimize the initiation of these sub workflows as optimization.

Additionally, it's not efficient because of the overhead of starting/signaling another workflow. It consume quite a lot of actions

A better way is:
* Extend the stateExecutionCounter to track the completed state executionId
* Use synchronous update with `workflow.Await( ()-> stateExecutionCounter.stateExecuctionCompleted(stateExeId) )` in the handler to wait for the execution

Note that continueAsNew need to be handled properly:
* In validator, it should check if the workflow has met continueAsNew threshold. If so, then return error
* In the `workflow.Await(...)` should also check continueAsNew threshold to return error so that workflow can do a continueAsNew
* The API service need to retry when continueAsNew interrupts the waiting
* returning state completion output will not be supported

Also, it doesn't allow reuse workflowId for different executions, because of the RunID issues in Temporal. This v2 can solve the problems

~~Note that sync update feature is not prod ready in Temporal yet. We will start working on it when it is ready.~~

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the current waitForStateCompletion implementation and the related sub-workflow approach in linked pull request #345, then trace stateExecutionCounter and the synchronous update entry point. Done means the v2 API handles continue-as-new interruption and retries, avoids the sub-workflow and RunID limitations, and does not return state completion output.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend-api-design, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.