argoproj / argoproj/argo-workflows
Parent Node Should Be Marked as Success With Recursive Template Success
- Dominant language
- Go
- Stars
- 17k
- Forks
- 3.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 138
Description
# Summary
- When a task within a DAG recursively calls its own template, the parent node’s status should reflect the last iteration in the recursive stack (i.e. the result of the last recursive template should propagate back up to the parent node).
- My purpose of a recursive template is to retry a specific section of the workflow.
- Therefore, it is more informative for the parent’s status to match the outcome of the final recursive template call rather than being marked as a failure due to an earlier failed retry that ultimately succeeded.
- The use of `continueOn` and `failFast` have not changed the behavior.
## Use Cases
- I am currently using a recursive template call to dynamically increase the memory allocated to a template whenever it detects an exit code of 137 (OOM error).
- Implementing this change means that if the third attempt with increased memory succeeds, the parent node will be marked as successful despite OOM failures in the previous two retry attempts.
- **NOTE:** The use of `retryStrategy` currently does not support dynamic memory increase with each retry while simultaneously retrying for transient errors.
---
**Message from the maintainers**:
Love this feature request? Give it a 👍. We prioritise the proposals with the most 👍.
---
## Example
Here is an example workflow that handles OOM errors by increasing memory allocation using recursive templates.
This is using Argo `v3.6.4`.
```yaml
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: oom-test-recursive
namespace: argo-workflows
spec:
entrypoint: start
serviceAccountName: s3-argo
templates:
- name: run-sb-stage
retryStrategy:
limit: "10"
retryPolicy: "Always"
expression: "lastRetry.status == 'Error'"
backoff:
duration: "1"
cap: "1m"
factor: 2
inputs:
parameters:
- name: memory
- name: cpu
value: 4
container:
image: python:3.9
command: ["python", "-c"]
args: ["s = ' ' * (1024 * 1024 * 1024 * 2)"] # Attempts to make a string of size 2 GB
podSpecPatch: '{"containers":[{"name":"main", "resources":{"limits":{"cpu": "{{inputs.parameters.cpu}}", "memory": "{{inputs.parameters.memory}}Mi"}}}]}'
- name: single-band-template
inputs:
parameters:
- name: multiplier
dag:
failFast: false
tasks:
- name: run-stage
template: run-sb-stage
arguments:
parameters:
- name: memory
value: "{{= 1000 * int(inputs.parameters.multiplier)}}" # Start with 1 GB (1000 MB)
# This template is only run if the run-stage template threw an OOM error
# This triggers the recursion
- name: oom-retry
template: single-band-template
arguments:
parameters:
- name: multiplier
value: "{{= int(inputs.parameters.multiplier) + 1}}"
depends: "run-stage.Failed"
when: "\"{{tasks.run-stage.exitCode}}\" == '137' && {{inputs.parameters.multiplier}} < 4"
# This template is only called if the run-stage template was successful
- name: process-next-stage
template: hello
depends: "run-stage.Succeeded"
- name: start
steps:
- - name: call-single-band
template: single-band-template
arguments:
parameters:
- name: multiplier
value: 1
# Placeholder template to simulate another template being called after the run-stage template
- name: hello
container:
image: alpine:3.6
command: [sh, -c]
args: ["echo \"Hello World\""]
```
---
Here is the result of the above workflow:

Ideally, the parent node should have a status of `Success` since the child recursive template succeeded in the end.
Contributor guide
Research direction
Start by reproducing the recursive-template workflow shown in the issue and trace how DAG node status is propagated through the recursive call stack. Done means the parent node reports the final recursive template outcome when a later retry succeeds, while the existing continueOn and failFast behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100