temporalio / temporalio/temporal
Allow to configure max concurrency of a workflow with the same id
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 23.2k
- Forks
- 1.9k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 228
Description
Is your feature request related to a problem? Please describe.
Hi, I'm curious if someone else run into that use case. We want to ensure that only one workflow with the same ID runs at the same time and that subsequent workflow runs are queued with a concurrency of 1. We are aware that this can be solved with SignalWithStartWorkflow and draining the signal channel but it is very cumbersome to implement. We need to build a queue our-self. We think this mechanism should be provided by temporal natively because it manages the task queue.
Also this approach does not create a new workflow run but extends the history. This is bad for traceability, We want to see a workflow run per job. Is there anything on the roadmap to cover this issue? Therefore are they other approaches to implement this result?
Describe the solution you'd like
Extending client.StartWorkflowOptions with a property e.g. TaskQueueConcurrency to control the desired concurrency.
workflowOptions := client.StartWorkflowOptions{
ID: "projects/my-branch"
TaskQueue: CreateDeploymentTaskQueueName,
WorkflowIDConcurrencyPolicy: enums.WORKFLOW_CONCURRENCY_POLICY_UNLIMITED | WORKFLOW_CONCURRENCY_POLICY_SERIAL,
}
we, err := c.ExecuteWorkflow(context.Background(), workflowOptions, CreateDeploymentWorkflow, options)
WORKFLOW_CONCURRENCY_POLICY_UNLIMITED: (default): Runs are scheduled without limits.WORKFLOW_CONCURRENCY_POLICY_SERIAL: Runs with the same ID are queued as separate runs and scheduled when the current workflow run is completed.
Describe alternatives you've considered
Using SignalWithStartWorkflow.
Additional context
https://temporalio.slack.com/archives/CTDTU3J4T/p1682069065643249
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing client.StartWorkflowOptions and the ExecuteWorkflow and SignalWithStartWorkflow entry points mentioned in the issue. Trace how workflow IDs and task queues currently handle concurrent runs, then determine where a serial concurrency policy would need to be represented. Done means separate runs with the same ID are queued and scheduled one at a time, while the default remains unlimited.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100