argoproj / argoproj/argo-workflows
`failFast` field doesn't work
- Dominant language
- Go
- Stars
- 17k
- Forks
- 3.7k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 138
Description
### Pre-requisites
- [X] I have double-checked my configuration
- [X] I can confirm the issues exists when I tested with `:latest`
- [ ] I'd like to contribute the fix myself (see [contributing guide](https://github.com/argoproj/argo-workflows/blob/master/docs/CONTRIBUTING.md))
### What happened/what you expected to happen?
failFast flag is invalid
### Version
v3.4.2
### Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
```YAML
I have a workflow yaml like this.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: dag-primay-branch-
spec:
entrypoint: statis
templates:
- name: a
container:
image: docker/whalesay:latest
command: [cowsay]
args: ["hello world"]
- name: b
retryStrategy:
limit: "2"
container:
image: alpine:latest
command: [sh, -c]
args: ["sleep 30; echo haha"]
- name: c
retryStrategy:
limit: "3"
container:
image: alpine:latest
command: [sh, -c]
args: ["echo intentional failure; exit 2"]
- name: d
container:
image: docker/whalesay:latest
command: [cowsay]
args: ["hello world"]
- name: statis
dag:
failFast: false
tasks:
- name: A
template: a
- name: B
depends: "A"
template: b
- name: C
depends: "A"
template: c
- name: D
depends: "B"
template: d
- name: E
depends: "D"
template: d
```
The dependencies are as follows :
```
step1: A
/ \
step2: B C
|
step3: D
|
step4: E
```
When I was using FailFast, I found that this parameter was invalid. Regardless of setting True/False, step C failed, and BDE would execute it.
```
### Logs from the workflow controller
```text
$ kubectl logs -n argo deploy/workflow-controller | grep dag-primay-branch-txrwg
time="2023-01-05T08:18:01.061Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:01.065Z" level=info msg="Updated phase -> Running" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:01.065Z" level=info msg="DAG node dag-primay-branch-txrwg initialized Running" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:01.065Z" level=info msg="All of node dag-primay-branch-txrwg.A dependencies [] completed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:01.065Z" level=info msg="Pod node dag-primay-branch-txrwg-4194302311 initialized Pending" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:01.087Z" level=info msg="Created pod: dag-primay-branch-txrwg.A (dag-primay-branch-txrwg-a-4194302311)" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:01.087Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:01.087Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:01.113Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534322 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:11.090Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:11.090Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:11.090Z" level=info msg="node changed" namespace=default new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=dag-primay-branch-txrwg-4194302311 old.message= old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:11.090Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:11.090Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:18:11.108Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534353 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.112Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.112Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.112Z" level=info msg="node changed" namespace=default new.message= new.phase=Succeeded new.progress=0/1 nodeID=dag-primay-branch-txrwg-4194302311 old.message=PodInitializing old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.112Z" level=info msg="All of node dag-primay-branch-txrwg.C dependencies [A] completed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.113Z" level=info msg="Retry node dag-primay-branch-txrwg-4227857549 initialized Running" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.113Z" level=info msg="Pod node dag-primay-branch-txrwg-3127743452 initialized Pending" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.136Z" level=info msg="Created pod: dag-primay-branch-txrwg.C(0) (dag-primay-branch-txrwg-c-3127743452)" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.136Z" level=info msg="All of node dag-primay-branch-txrwg.B dependencies [A] completed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.136Z" level=info msg="Retry node dag-primay-branch-txrwg-4211079930 initialized Running" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.136Z" level=info msg="Pod node dag-primay-branch-txrwg-1127430305 initialized Pending" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.157Z" level=info msg="Created pod: dag-primay-branch-txrwg.B(0) (dag-primay-branch-txrwg-b-1127430305)" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.157Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.157Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.196Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534522 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:33.202Z" level=info msg="cleaning up pod" action=labelPodCompleted key=default/dag-primay-branch-txrwg-a-4194302311/labelPodCompleted
time="2023-01-05T08:19:43.138Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:43.138Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:43.138Z" level=info msg="node changed" namespace=default new.message= new.phase=Running new.progress=0/1 nodeID=dag-primay-branch-txrwg-1127430305 old.message= old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:43.138Z" level=info msg="node changed" namespace=default new.message= new.phase=Running new.progress=0/1 nodeID=dag-primay-branch-txrwg-3127743452 old.message= old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:43.139Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:43.139Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:43.144Z" level=info msg="cleaning up pod" action=terminateContainers key=default/dag-primay-branch-txrwg-c-3127743452/terminateContainers
time="2023-01-05T08:19:43.151Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534582 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.223Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.223Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.224Z" level=info msg="node unchanged" namespace=default nodeID=dag-primay-branch-txrwg-1127430305 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.224Z" level=info msg="node changed" namespace=default new.message="Error (exit code 2)" new.phase=Failed new.progress=0/1 nodeID=dag-primay-branch-txrwg-3127743452 old.message= old.phase=Running old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.224Z" level=info msg="1 child nodes of dag-primay-branch-txrwg.C failed. Trying again..." namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.224Z" level=info msg="Pod node dag-primay-branch-txrwg-2523602073 initialized Pending" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.236Z" level=info msg="Created pod: dag-primay-branch-txrwg.C(1) (dag-primay-branch-txrwg-c-2523602073)" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.236Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.236Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.249Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534603 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:19:53.255Z" level=info msg="cleaning up pod" action=labelPodCompleted key=default/dag-primay-branch-txrwg-c-3127743452/labelPodCompleted
time="2023-01-05T08:20:03.240Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.241Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.241Z" level=info msg="node changed" namespace=default new.message="Error (exit code 2)" new.phase=Failed new.progress=0/1 nodeID=dag-primay-branch-txrwg-2523602073 old.message= old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.241Z" level=info msg="node unchanged" namespace=default nodeID=dag-primay-branch-txrwg-1127430305 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.241Z" level=info msg="2 child nodes of dag-primay-branch-txrwg.C failed. Trying again..." namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.241Z" level=info msg="Pod node dag-primay-branch-txrwg-1047024506 initialized Pending" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.266Z" level=info msg="Created pod: dag-primay-branch-txrwg.C(2) (dag-primay-branch-txrwg-c-1047024506)" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.266Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.266Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.289Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534649 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:03.295Z" level=info msg="cleaning up pod" action=labelPodCompleted key=default/dag-primay-branch-txrwg-c-2523602073/labelPodCompleted
time="2023-01-05T08:20:13.145Z" level=info msg="cleaning up pod" action=killContainers key=default/dag-primay-branch-txrwg-c-3127743452/killContainers
time="2023-01-05T08:20:13.268Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.268Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.268Z" level=info msg="node changed" namespace=default new.message= new.phase=Running new.progress=0/1 nodeID=dag-primay-branch-txrwg-1127430305 old.message= old.phase=Running old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.268Z" level=info msg="node changed" namespace=default new.message="Error (exit code 2)" new.phase=Failed new.progress=0/1 nodeID=dag-primay-branch-txrwg-1047024506 old.message= old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.268Z" level=info msg="3 child nodes of dag-primay-branch-txrwg.C failed. Trying again..." namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.268Z" level=info msg="Pod node dag-primay-branch-txrwg-442883127 initialized Pending" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.273Z" level=info msg="cleaning up pod" action=terminateContainers key=default/dag-primay-branch-txrwg-b-1127430305/terminateContainers
time="2023-01-05T08:20:13.281Z" level=info msg="Created pod: dag-primay-branch-txrwg.C(3) (dag-primay-branch-txrwg-c-442883127)" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.281Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.281Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.304Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534705 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:13.311Z" level=info msg="cleaning up pod" action=labelPodCompleted key=default/dag-primay-branch-txrwg-c-1047024506/labelPodCompleted
time="2023-01-05T08:20:23.283Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="node changed" namespace=default new.message= new.phase=Succeeded new.progress=0/1 nodeID=dag-primay-branch-txrwg-1127430305 old.message= old.phase=Running old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="node changed" namespace=default new.message="Error (exit code 2)" new.phase=Failed new.progress=0/1 nodeID=dag-primay-branch-txrwg-442883127 old.message= old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="No more retries left. Failing..." namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="node dag-primay-branch-txrwg-4227857549 phase Running -> Failed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="node dag-primay-branch-txrwg-4227857549 message: No more retries left" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="node dag-primay-branch-txrwg-4227857549 finished: 2023-01-05 08:20:23.284703468 +0000 UTC" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="node dag-primay-branch-txrwg-4211079930 phase Running -> Succeeded" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="node dag-primay-branch-txrwg-4211079930 finished: 2023-01-05 08:20:23.284844065 +0000 UTC" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.284Z" level=info msg="All of node dag-primay-branch-txrwg.D dependencies [B] completed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.285Z" level=info msg="Pod node dag-primay-branch-txrwg-4110414216 initialized Pending" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.297Z" level=info msg="Created pod: dag-primay-branch-txrwg.D (dag-primay-branch-txrwg-d-4110414216)" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.297Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.297Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.317Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534761 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:23.323Z" level=info msg="cleaning up pod" action=labelPodCompleted key=default/dag-primay-branch-txrwg-b-1127430305/labelPodCompleted
time="2023-01-05T08:20:23.323Z" level=info msg="cleaning up pod" action=labelPodCompleted key=default/dag-primay-branch-txrwg-c-442883127/labelPodCompleted
time="2023-01-05T08:20:33.299Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:33.299Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:33.299Z" level=info msg="node changed" namespace=default new.message= new.phase=Running new.progress=0/1 nodeID=dag-primay-branch-txrwg-4110414216 old.message= old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:33.299Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:33.299Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:33.304Z" level=info msg="cleaning up pod" action=terminateContainers key=default/dag-primay-branch-txrwg-d-4110414216/terminateContainers
time="2023-01-05T08:20:33.320Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534803 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.274Z" level=info msg="cleaning up pod" action=killContainers key=default/dag-primay-branch-txrwg-b-1127430305/killContainers
time="2023-01-05T08:20:43.647Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.647Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.647Z" level=info msg="node changed" namespace=default new.message= new.phase=Succeeded new.progress=0/1 nodeID=dag-primay-branch-txrwg-4110414216 old.message= old.phase=Running old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.648Z" level=info msg="All of node dag-primay-branch-txrwg.E dependencies [D] completed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.648Z" level=info msg="Pod node dag-primay-branch-txrwg-4127191835 initialized Pending" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.665Z" level=info msg="Created pod: dag-primay-branch-txrwg.E (dag-primay-branch-txrwg-d-4127191835)" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.665Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.665Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.686Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534827 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:43.692Z" level=info msg="cleaning up pod" action=labelPodCompleted key=default/dag-primay-branch-txrwg-d-4110414216/labelPodCompleted
time="2023-01-05T08:20:53.667Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:53.667Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:53.667Z" level=info msg="node changed" namespace=default new.message= new.phase=Running new.progress=0/1 nodeID=dag-primay-branch-txrwg-4127191835 old.message= old.phase=Pending old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:53.671Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:53.671Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:20:53.673Z" level=info msg="cleaning up pod" action=terminateContainers key=default/dag-primay-branch-txrwg-d-4127191835/terminateContainers
time="2023-01-05T08:20:53.691Z" level=info msg="Workflow update successful" namespace=default phase=Running resourceVersion=2534863 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.305Z" level=info msg="cleaning up pod" action=killContainers key=default/dag-primay-branch-txrwg-d-4110414216/killContainers
time="2023-01-05T08:21:03.788Z" level=info msg="Processing workflow" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="Task-result reconciliation" namespace=default numObjs=0 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="node changed" namespace=default new.message= new.phase=Succeeded new.progress=0/1 nodeID=dag-primay-branch-txrwg-4127191835 old.message= old.phase=Running old.progress=0/1 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="Outbound nodes of dag-primay-branch-txrwg set to [dag-primay-branch-txrwg-442883127 dag-primay-branch-txrwg-4127191835]" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="node dag-primay-branch-txrwg phase Running -> Failed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="node dag-primay-branch-txrwg finished: 2023-01-05 08:21:03.789608909 +0000 UTC" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="Checking daemoned children of dag-primay-branch-txrwg" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="TaskSet Reconciliation" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg=reconcileAgentPod namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="Updated phase Running -> Failed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="Marking workflow completed" namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.789Z" level=info msg="Checking daemoned children of " namespace=default workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.795Z" level=info msg="cleaning up pod" action=deletePod key=default/dag-primay-branch-txrwg-1340600742-agent/deletePod
time="2023-01-05T08:21:03.812Z" level=info msg="Workflow update successful" namespace=default phase=Failed resourceVersion=2534886 workflow=dag-primay-branch-txrwg
time="2023-01-05T08:21:03.826Z" level=info msg="cleaning up pod" action=labelPodCompleted key=default/dag-primay-branch-txrwg-d-4127191835/labelPodCompleted
time="2023-01-05T08:21:23.674Z" level=info msg="cleaning up pod" action=killContainers key=default/dag-primay-branch-txrwg-d-4127191835/killContainers
```
### Logs from in your workflow's wait container
```text
$ kubectl logs -c wait -l workflows.argoproj.io/workflow="dag-primay-branch-txrwg",workflow.argoproj.io/phase!=Succeeded
time="2023-01-05T08:18:03.985Z" level=info msg="Starting Workflow Executor" version=v3.4.2
time="2023-01-05T08:18:03.987Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-01-05T08:18:03.987Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=default podName=dag-primay-branch-txrwg-a-4194302311 template="{\"name\":\"a\",\"inputs\":{},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"docker/whalesay:latest\",\"command\":[\"cowsay\"],\"args\":[\"hello world\"],\"resources\":{}}}" version="&Version{Version:v3.4.2,BuildDate:2022-10-23T04:49:38Z,GitCommit:c08563baf7bafafe3aeb3284cd3410308603cad4,GitTag:v3.4.2,GitTreeState:clean,GoVersion:go1.18.7,Compiler:gc,Platform:linux/amd64,}"
time="2023-01-05T08:18:03.987Z" level=info msg="Starting deadline monitor"
time="2023-01-05T08:19:24.036Z" level=info msg="Main container completed" error=""
time="2023-01-05T08:19:24.036Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-01-05T08:19:24.036Z" level=info msg="No output parameters"
time="2023-01-05T08:19:24.036Z" level=info msg="No output artifacts"
time="2023-01-05T08:19:24.036Z" level=info msg="Alloc=6736 TotalAlloc=12007 Sys=18898 NumGC=4 Goroutines=7"
time="2023-01-05T08:19:36.448Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-01-05T08:19:36.448Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=default podName=dag-primay-branch-txrwg-c-3127743452 template="{\"name\":\"c\",\"inputs\":{},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"alpine:latest\",\"command\":[\"sh\",\"-c\"],\"args\":[\"echo intentional failure; exit 2\"],\"resources\":{}},\"retryStrategy\":{\"limit\":3}}" version="&Version{Version:v3.4.2,BuildDate:2022-10-23T04:49:38Z,GitCommit:c08563baf7bafafe3aeb3284cd3410308603cad4,GitTag:v3.4.2,GitTreeState:clean,GoVersion:go1.18.7,Compiler:gc,Platform:linux/amd64,}"
time="2023-01-05T08:19:36.449Z" level=info msg="Starting deadline monitor"
time="2023-01-05T08:19:40.451Z" level=info msg="Main container completed" error=""
time="2023-01-05T08:19:40.451Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-01-05T08:19:40.451Z" level=info msg="No output parameters"
time="2023-01-05T08:19:40.451Z" level=info msg="No output artifacts"
time="2023-01-05T08:19:40.451Z" level=info msg="stopping progress monitor (context done)" error="context canceled"
time="2023-01-05T08:19:40.451Z" level=info msg="Deadline monitor stopped"
time="2023-01-05T08:19:40.451Z" level=info msg="Alloc=7969 TotalAlloc=12272 Sys=18642 NumGC=3 Goroutines=4"
time="2023-01-05T08:19:55.536Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-01-05T08:19:55.537Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=default podName=dag-primay-branch-txrwg-c-2523602073 template="{\"name\":\"c\",\"inputs\":{},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"alpine:latest\",\"command\":[\"sh\",\"-c\"],\"args\":[\"echo intentional failure; exit 2\"],\"resources\":{}},\"retryStrategy\":{\"limit\":3}}" version="&Version{Version:v3.4.2,BuildDate:2022-10-23T04:49:38Z,GitCommit:c08563baf7bafafe3aeb3284cd3410308603cad4,GitTag:v3.4.2,GitTreeState:clean,GoVersion:go1.18.7,Compiler:gc,Platform:linux/amd64,}"
time="2023-01-05T08:19:55.537Z" level=info msg="Starting deadline monitor"
time="2023-01-05T08:19:58.538Z" level=info msg="Main container completed" error=""
time="2023-01-05T08:19:58.538Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-01-05T08:19:58.538Z" level=info msg="No output parameters"
time="2023-01-05T08:19:58.538Z" level=info msg="No output artifacts"
time="2023-01-05T08:19:58.538Z" level=info msg="stopping progress monitor (context done)" error="context canceled"
time="2023-01-05T08:19:58.538Z" level=info msg="Deadline monitor stopped"
time="2023-01-05T08:19:58.539Z" level=info msg="Alloc=6415 TotalAlloc=12236 Sys=19666 NumGC=4 Goroutines=5"
time="2023-01-05T08:20:05.584Z" level=info msg="Starting Workflow Executor" version=v3.4.2
time="2023-01-05T08:20:05.589Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-01-05T08:20:05.589Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=default podName=dag-primay-branch-txrwg-c-1047024506 template="{\"name\":\"c\",\"inputs\":{},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"alpine:latest\",\"command\":[\"sh\",\"-c\"],\"args\":[\"echo intentional failure; exit 2\"],\"resources\":{}},\"retryStrategy\":{\"limit\":3}}" version="&Version{Version:v3.4.2,BuildDate:2022-10-23T04:49:38Z,GitCommit:c08563baf7bafafe3aeb3284cd3410308603cad4,GitTag:v3.4.2,GitTreeState:clean,GoVersion:go1.18.7,Compiler:gc,Platform:linux/amd64,}"
time="2023-01-05T08:20:05.589Z" level=info msg="Starting deadline monitor"
time="2023-01-05T08:20:08.589Z" level=info msg="Main container completed" error=""
time="2023-01-05T08:20:08.589Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-01-05T08:20:08.589Z" level=info msg="No output parameters"
time="2023-01-05T08:20:08.589Z" level=info msg="No output artifacts"
time="2023-01-05T08:20:08.589Z" level=info msg="stopping progress monitor (context done)" error="context canceled"
time="2023-01-05T08:20:08.589Z" level=info msg="Alloc=6368 TotalAlloc=12222 Sys=19154 NumGC=4 Goroutines=7"
time="2023-01-05T08:20:15.595Z" level=info msg="Starting Workflow Executor" version=v3.4.2
time="2023-01-05T08:20:15.599Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-01-05T08:20:15.599Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=default podName=dag-primay-branch-txrwg-c-442883127 template="{\"name\":\"c\",\"inputs\":{},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"alpine:latest\",\"command\":[\"sh\",\"-c\"],\"args\":[\"echo intentional failure; exit 2\"],\"resources\":{}},\"retryStrategy\":{\"limit\":3}}" version="&Version{Version:v3.4.2,BuildDate:2022-10-23T04:49:38Z,GitCommit:c08563baf7bafafe3aeb3284cd3410308603cad4,GitTag:v3.4.2,GitTreeState:clean,GoVersion:go1.18.7,Compiler:gc,Platform:linux/amd64,}"
time="2023-01-05T08:20:15.599Z" level=info msg="Starting deadline monitor"
time="2023-01-05T08:20:18.599Z" level=info msg="Main container completed" error=""
time="2023-01-05T08:20:18.599Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-01-05T08:20:18.599Z" level=info msg="No output parameters"
time="2023-01-05T08:20:18.599Z" level=info msg="No output artifacts"
time="2023-01-05T08:20:18.599Z" level=info msg="Deadline monitor stopped"
time="2023-01-05T08:20:18.599Z" level=info msg="Alloc=6222 TotalAlloc=12188 Sys=23506 NumGC=4 Goroutines=7"
time="2023-01-05T08:19:36.457Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-01-05T08:19:36.457Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=default podName=dag-primay-branch-txrwg-b-1127430305 template="{\"name\":\"b\",\"inputs\":{},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"alpine:latest\",\"command\":[\"sh\",\"-c\"],\"args\":[\"sleep 30; echo haha\"],\"resources\":{}},\"retryStrategy\":{\"limit\":2}}" version="&Version{Version:v3.4.2,BuildDate:2022-10-23T04:49:38Z,GitCommit:c08563baf7bafafe3aeb3284cd3410308603cad4,GitTag:v3.4.2,GitTreeState:clean,GoVersion:go1.18.7,Compiler:gc,Platform:linux/amd64,}"
time="2023-01-05T08:19:36.457Z" level=info msg="Starting deadline monitor"
time="2023-01-05T08:20:10.473Z" level=info msg="Main container completed" error=""
time="2023-01-05T08:20:10.473Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-01-05T08:20:10.473Z" level=info msg="No output parameters"
time="2023-01-05T08:20:10.473Z" level=info msg="No output artifacts"
time="2023-01-05T08:20:10.473Z" level=info msg="Deadline monitor stopped"
time="2023-01-05T08:20:10.473Z" level=info msg="stopping progress monitor (context done)" error="context canceled"
time="2023-01-05T08:20:10.473Z" level=info msg="Alloc=6103 TotalAlloc=12035 Sys=19154 NumGC=4 Goroutines=5"
time="2023-01-05T08:20:25.689Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-01-05T08:20:25.689Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=default podName=dag-primay-branch-txrwg-d-4110414216 template="{\"name\":\"d\",\"inputs\":{},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"docker/whalesay:latest\",\"command\":[\"cowsay\"],\"args\":[\"hello world\"],\"resources\":{}}}" version="&Version{Version:v3.4.2,BuildDate:2022-10-23T04:49:38Z,GitCommit:c08563baf7bafafe3aeb3284cd3410308603cad4,GitTag:v3.4.2,GitTreeState:clean,GoVersion:go1.18.7,Compiler:gc,Platform:linux/amd64,}"
time="2023-01-05T08:20:25.690Z" level=info msg="Starting deadline monitor"
time="2023-01-05T08:20:31.693Z" level=info msg="Main container completed" error=""
time="2023-01-05T08:20:31.693Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-01-05T08:20:31.693Z" level=info msg="No output parameters"
time="2023-01-05T08:20:31.693Z" level=info msg="No output artifacts"
time="2023-01-05T08:20:31.693Z" level=info msg="Deadline monitor stopped"
time="2023-01-05T08:20:31.693Z" level=info msg="stopping progress monitor (context done)" error="context canceled"
time="2023-01-05T08:20:31.693Z" level=info msg="Alloc=6874 TotalAlloc=12123 Sys=18898 NumGC=3 Goroutines=4"
time="2023-01-05T08:20:45.802Z" level=info msg="Starting Workflow Executor" version=v3.4.2
time="2023-01-05T08:20:45.807Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2023-01-05T08:20:45.807Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=default podName=dag-primay-branch-txrwg-d-4127191835 template="{\"name\":\"d\",\"inputs\":{},\"outputs\":{},\"metadata\":{},\"container\":{\"name\":\"\",\"image\":\"docker/whalesay:latest\",\"command\":[\"cowsay\"],\"args\":[\"hello world\"],\"resources\":{}}}" version="&Version{Version:v3.4.2,BuildDate:2022-10-23T04:49:38Z,GitCommit:c08563baf7bafafe3aeb3284cd3410308603cad4,GitTag:v3.4.2,GitTreeState:clean,GoVersion:go1.18.7,Compiler:gc,Platform:linux/amd64,}"
time="2023-01-05T08:20:45.807Z" level=info msg="Starting deadline monitor"
time="2023-01-05T08:20:51.807Z" level=info msg="Main container completed" error=""
time="2023-01-05T08:20:51.807Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2023-01-05T08:20:51.807Z" level=info msg="No output parameters"
time="2023-01-05T08:20:51.807Z" level=info msg="No output artifacts"
time="2023-01-05T08:20:51.807Z" level=info msg="Deadline monitor stopped"
time="2023-01-05T08:20:51.807Z" level=info msg="Alloc=6129 TotalAlloc=12208 Sys=19154 NumGC=4 Goroutines=6"
```
Contributor guide
Research direction
Start by reproducing the supplied Workflow with the workflow controller and compare DAG execution when failFast is true versus false. Trace the controller's DAG failFast handling using the provided logs; done means the setting produces distinct, correct execution behavior for the failing C branch and dependent B-D-E tasks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100