argoproj / argoproj/argo-workflows

Make sure the output directories always exist by mounting volumes there if needed

Open
#4,096 1 comment 0 reactions 0 assignees View on GitHub
area/executor
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

# Summary

This is just a small usability improvement.

Currently, the the code running in Argo-initiated containers needs to create parent directories before writing outputs.

For some executors Argo already mounts volumes under the output paths to capture the outputs. We could make this standard and ensure all outputs have volumes mounted under them.

# Use Cases

This is mainly useful when using command-line programs that only support outputting files to existing directories (most command-line programs are like that). Sure, the user can always write a wrapper script, but that is boilerplate.

Compare:
```yaml
command:
- tensorflowjs_converter
- --input_format=tf_saved_model
- --output_format=tfjs_graph_model
- /tmp/inputs/Model/data
- /tmp/outputs/Model/data
```
vs.

```yaml
command:
- sh
- -exc
- |
model_path="$0"
output_model_path="$1"
mkdir -p "$(dirname "$output_model_path")"

tflite_convert --saved_model_dir "$model_path" --output_file "$output_model_path"
- /tmp/inputs/Model/data
- /tmp/outputs/Model/data
```

---

**Message from the maintainers**:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

Contributor guide

Open the contributing guide

Research direction

Start by tracing how Argo-initiated containers and executors mount volumes for captured outputs. The work is done when every output path has a suitable volume mounted, so command-line programs can write to nested output paths without creating parent directories themselves.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.