argoproj / argoproj/argo-workflows

Mount volumes defined in a WorkflowTemplate when it's called by another workflow

Open
#7,444 7 comments 3 reactions 0 assignees View on GitHub
area/workflow-templates
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

# Summary

### What change needs making?

Volumes set on a WorkflowTemplate should be used when called by another workflow

# Use Cases

### When would you use this?

Here is what doesn't work today

```yaml
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: template1
spec:
entrypoint: template1
volumeClaimTemplates:
- metadata:
name: my-pvc
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5M
arguments: ¶meters
parameters:
- name: run
templates:
- name: template1
inputs: *parameters
steps:
- - name: template10
template: template10
arguments:
parameters:
- name: run
value: "{{inputs.parameters.run}}"
- - name: template11
templateRef:
name: template2
template: template2

- name: template10
inputs: *parameters
script:
image: debian:9.4
command: [bash]
source: |
echo "{{inputs.parameters.run}}" > /mnt/pvc/output
volumeMounts:
- name: my-pvc
mountPath: /mnt/pvc

---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: template2
spec:
entrypoint: template2
templates:
- name: template2
script:
image: debian:9.4
command: [bash]
source: |
cat /mnt/pvc/output
volumeMounts:
- name: my-pvc
mountPath: /mnt/pvc
---
apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
name: cronwf-ex1
spec:
schedule: 0 11 * * *
concurrencyPolicy: Forbid
workflowSpec:
entrypoint: jobs
# volumeClaimTemplates:
# - metadata:
# name: my-pvc
# spec:
# accessModes: [ "ReadWriteOnce" ]
# resources:
# requests:
# storage: 5M
templates:
- name: jobs
steps:
- - name: template1
templateRef:
name: template1
template: template1
arguments:
parameters:
- name: run
value: "{{item}}"
withItems:
- run1
# -...
- run23
```

**works well**
_argo submit --from wftmpl/template1 --watch_

**works only if the commented out lines on the cron workflow are uncommented**
_argo submit --from cronwf/cronwf-ex1 --watch_

The issue with adding the volume on the cronworkflow is that it's shared by all the runX however I want to keep it in the template1 to make sure each runX uses a different PVC (gp2 can't use ReadWriteMany for instance).

volumes available on the workflow template should be automatically mounted when called by another template. It could be not mounted only if the calling workflow already has a corresponding volume for instance.

---

**Message from the maintainers**:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

Contributor guide

Open the contributing guide

Research direction

Reproduce the difference between `argo submit --from wftmpl/template1 --watch` and `argo submit --from cronwf/cronwf-ex1 --watch` using the supplied WorkflowTemplate and CronWorkflow manifests. Trace how a `templateRef` call resolves `volumeClaimTemplates`, then define and test the behavior for per-call PVCs and conflicts with volumes on the calling workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.