argoproj / argoproj/argo-workflows
Proposal: Count-based Enhanced Depends Logic
- Dominant language
- Go
- Stars
- 17k
- Forks
- 3.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 138
Description
# Summary
Support count-based enhanced depends logic in addition to existing depends logic based on dependent tasks and their statuses as outlined in the [Enhanced Depends Logic](https://github.com/argoproj/argo/blob/master/docs/enhanced-depends-logic.md) document that has been introduced after v2.9.
# Motivation
Currently, there exists a new called `depends` that allows users to specify dependent tasks, their statuses, as well as any complex boolean logic. The field is a `string` field and the syntax is expression-like with operands having the form `.`. Examples include `task-1.Suceeded`, `task-2.Failed`, and `task-3.Damenoed`. More details on this existing functionality can be found in the [Enhanced Depends Logic](https://github.com/argoproj/argo/blob/master/docs/enhanced-depends-logic.md) document.
However, this requires users to specify task names explicitly which may be difficult or unnecessary to obtain, for example:
* There are situations where the tasks are dynamically generated and task names cannot be easily obtained.
* Some users might not care the statuses for specific tasks and instead focus on the number of tasks of a particular status.
For example, some users might want to trigger the downstream task when N out of M dependent tasks have succeeded statuses. They may use Argo for use cases where specific statuses of the dependent tasks are not important.
# Proposal
Extend the current `depends` field to be able to take complex comparison logic with operands having the form `` without task names.
For example, the following `depends` string indicates that we only execute downstream tasks when at least 3 tasks succeeded and no task failed:
```
depends: "Succeeded > 3 && Failed == 0"
```
Another example is to have a built-in `N` in the implementation that represents the total number of dependent tasks so users do not need to know the total number of dependent tasks beforehand.
A simple case is only executing downstream tasks when all dependent tasks succeeded:
```
depends: "Succeeded == N"
```
Users should be able to do simple calculations like the following:
```
depends: "Succeeded > N - 10 && Failed < N"
```
The full list of available task results and is as follows:
| Task Result | Description |
|:------------:|----------------|
| `Succeeded` | Task Succeeded |
| `Failed` | Task Failed |
| `Errored` | Task Errored |
| `Skipped` | Task Skipped |
| `Completed` | Task Succeded or Failed |
| `Daemoned` | Task is Daemoned and is not Pending |
Full boolean logic is available:
* `&&`
* `||`
* `!`
The full list of available math operators that we plan to support initially include:
* `+`
* `-`
* `*`
* `/`
## Compatibility
Because of the added control in `depends` is based on count. `dag.task.continueOn` is not available when using it. It is not possible to use both `dependencies` and `depends` in the same task group. Furthermore, users should only be able to use one of the two types of enhanced depends logic: count-based or task-name-based.
---
**Message from the maintainers**:
If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.
Contributor guide
Research direction
Start with the Enhanced Depends Logic document linked in the issue and trace the existing `depends` field implementation and parsing entry points. Define how count operands and arithmetic interact with the existing task-name syntax, then verify the stated compatibility rules for `dependencies` and `continueOn`. Done means count-based expressions support the listed results, operators, and examples without breaking existing depends logic.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100