Support directories in `artifact_paths`
- Dominant language
- Go
- Stars
- 1.1k
- Forks
- 378
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 74
Description
Suppose I want to upload all contents of a directory `foo/bar`. I would love to specify artifact paths like this:
```yaml
artifact_paths:
- foo/bar
```
I would expect this to be treated the same as `foo/bar/*`. But actually no artifacts are uploaded, even if the directory exists and is non-empty. This is further complicated by the name of the variable/key. The problem is that`artifact_paths` are not paths, they are _patterns_. It is even more confusing that they do not support directories which _are_ paths!
Suppose I have a command step that wants to create one directory from `$BUILDKITE_ARTIFACT_PATHS` if it doesn't exist. How would I go about it without first-class directory support?
Well, let's try the common shell script and assume we have the following glob pattern configured:
```yaml
artifact_paths:
- foo/**/*
```
In our command:
```ShellSession
# Try to create directory foo
mkdir $BUILDKITE_ARTIFACTS_PATHS
```
... oops, now we created a directory called `foo/**/*`, what I really wanted was to create a directory called `foo` and put all my artifacts in that directory.
This is even _more_ confusing as a user because of the existence of `nullglob`, `dotglob` and the other variety of shell options. Maybe we should all stop using shell scripts (I don't disagree). But I think for now, it would be better to support directories in `artifact_paths`.
**Describe the solution you'd like**
If a path `foo` is specified in `artifact_paths` and it is a directory that exists, it should be treated like `foo/*` during the upload step.
Edit: I think the directory should be treated like `foo/**/*`
Contributor guide
Research direction
Start by locating the Go code that parses artifact_paths and performs artifact uploads, then find its related tests and existing glob-pattern handling. Verify how an existing directory is detected and define completion as uploading its contents recursively, matching the requested foo/**/* behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- ci-cd, devops
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100