argoproj / argoproj/argo-workflows

Add support for more granular and flexible synchronization mechanism of the workflows

Open
#7,272 2 comments 24 reactions 0 assignees View on GitHub
area/mutex-semaphore area/spec
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

# Summary

Add support for more granular and flexible synchronization mechanism of the workflow

# Use Cases

Today you can enforce your workflow concurrency only by 2 levels: workflow or template.
The proposal is to add an ability to create custom levels of synchronization based on workflow fields like labels or parameter values.

The interface can be the following:
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: synchronization-wf-level-
spec:
entrypoint: whalesay
synchronization:
semaphore:
configMapKeyRef:
name: my-config
key: workflow
parameters:
- name: repo
value: '{{ workflow.parameters.GIT_REPO }}'
- name: sensor
value: '{{ workflow.lables.events.argoproj.io/sensor }}'
templates:
- name: whalesay
container:
image: docker/whalesay:latest
command: [cowsay]
args: ["hello world"]
```

It should also fit well into the current implementation of semaphore synchronization.
I have checked and it looks like currently semaphore lock names are created in this way:
```go
fmt.Sprintf("%s/%s/%s/%s", ln.Namespace, ln.Kind, ln.ResourceName, ln.Key)
```
We can add an additional variable here that will hold the value of provided parameters concatenated together:
```go
// ln.Parameters = "repo=argoproj/argo-workflows&sensor=sensor-name"
fmt.Sprintf("%s/%s/%s/%s/%s", ln.Namespace, ln.Kind, ln.ResourceName, ln.Key, ln.Parameters)
```

I would like to submit a PR if you think it's good to have these synchronization capabilities.

---

**Message from the maintainers**:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing semaphore synchronization implementation and the lock-name construction shown in the issue. Review the Workflow synchronization schema and determine how configured parameters should be resolved into lock identity. Done means custom synchronization levels work from workflow fields or parameter values while preserving current workflow- and template-level semaphore behavior, with tests for the new configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.