temporalio / temporalio/temporal

Continue workflow as new and signal in a single transaction

Open
#3,008 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement feature-request
Dominant language
Go
Stars
23.2k
Forks
1.9k
Avg merge
2d 8h
Merged PRs (30d)
228

Description

I will be using Go SDK naming here. But I believe this is a server-side feature request.

Is your feature request related to a problem? Please describe.

It is possible to start a new workflow and send a signal to it in a signal transaction using SignalWithStartWorkflow. However, it is not possible to continue a workflow as new (NewContinueAsNewError) and send a signal to it in a single transaction.

A workflow example that demonstrates why this might be useful:

  • The workflow is running "an infinite loop" by returning NewContinueAsNewError at the end of each execution.
  • During each iteration of this workflow it runs some activities given an input. This input can be updated while the workflow is running. The workflow will finish the current iteration using the old input. The updated input is used on the next iteration.
  • In fact, the input can be updated multiple times during the single iteration. The next iteration must use the most recent input. All other input is considered stale and is discarded.
  • The workflow sleeps at the end of each iteration if no new input was provided and starts the next iteration using the old input.
  • If new input was provided while the workflow was running activities/during sleep, it skips sleep/stop sleeping and starts the next iteration using the new input right away.

This description suggests that it's best to provide input as a signal.

Pseudocode to implement this workflow:

  1. Start workflow/signal already running workflow using SignalWithStartWorkflow
  2. Receive input signal
  3. Long running activities that use input
  4. Sleep unless new input is received:
    4.1. No new input. Sleep until timer finishes. Return NewContinueAsNewError and signal old input in the same transaction.
    4.2 New input was signalled (maybe several signals). Drain signal channel to get the most recently provided input. Return NewContinueAsNewError and signal new input in a single transaction.

Describe the solution you'd like

Add new error type that is like NewContinueAsNewError but allows signalling the new workflow in a single transaction like SignalWithStartWorkflow.

Describe alternatives you've considered

Right now I wrote my workflow to accept the same input struct using both workflow function argument and signal. I need to accept input as signals because of the requirement to be able to update input for the next iteration while the current iteration is running. I need to be able to provide input as argument since this is currently the only way to provide input to the workflow that is started using NewContinueAsNewError.

Providing the same input type using both workflow argument and signal is awkward, thus my feature request.

Additional context

In case my use case description and pseudocode don't make sense, here's the skeleton implementation of the workflow that I currently have: https://gist.github.com/mmxmb/d3a2b490855c98f0a28f3f6d30a005e2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing SignalWithStartWorkflow and NewContinueAsNewError behavior, then compare it with the linked workflow skeleton. Define how continuing as new and signaling the next execution would be atomic, including which input should be delivered when several signals arrive. Done means the server-side feature is implemented with coverage for the described workflow cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.