Support default convention for dynamic pipeline generation
- Dominant language
- Go
- Stars
- 1.1k
- Forks
- 378
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 74
Description
**Is your feature request related to a problem? Please describe.**
I want to generate a pipeline dynamically but don't want to change the default pipeline defined in the Buildkite UI and don't want extra build steps for every build just to generate/upload dynamic pipelines.
**Describe the solution you'd like**
We show an example in the docs of:
```
$ ./script/dynamic_step_generator | buildkite-agent pipeline upload
```
My request is to have a conventional default for that behavior rather than requiring a custom pipeline to invoke it.
Add a default "dynamic pipeline generator script" to the [list of pipeline files](https://buildkite.com/docs/agent/v3/cli-pipeline#uploading-pipelines-description) that the buildkite-agent automatically detects.
Specific name doesn't matter but `buildkite.sh` / `.buildkite/pipeline.sh` seems reasonable.
When `buildkite-agent pipeline upload` is run and a `buildkite.sh` or `.buildkite/pipeline.sh` file exists and is executable, execute it and take the output as a pipeline.
**Describe alternatives you've considered**
* Living with having one CI job step to "upload pipeline" and then another immediately after to "generate and upload dynamic pipeline"
* Editing each project in the Buildkite UI where we use dynamic pipelines to have a different default pipeline. This doesn't feel great for us as we really want the pipeline logic to fully live in the git repo.
**Additional context**
N/A
Contributor guide
Research direction
Start at the `buildkite-agent pipeline upload` entry point and read the pipeline-uploading documentation linked in the issue. Investigate how `buildkite.sh` and `.buildkite/pipeline.sh` could be detected and invoked only when executable; done means their output is accepted as the pipeline without requiring an extra configured build step.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- ci-cd, cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100