buildkite / buildkite/agent

Support for artifact configuration at the pipeline step level

Open
#1,794 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1.1k
Forks
378
Avg merge
2d 6h
Merged PRs (30d)
74

Description

**Is your feature request related to a problem? Please describe.**
We run multiple pipelines on the same hosts. We also don't allow hooks to be overwritten at runtime as per the security recommendations here: https://buildkite.com/docs/agent/v3/securing#disabling-local-hooks. This leaves us with only one choice for specifying where to upload artifacts: global hooks configuration. This approach does not work well for us because we have multiple pipelines with different requirements. We either need to do some plumbing at the environment level and check what pipeline is being called and based on that to export the appropriate configuration, but it is rather cumbersome.

**Describe the solution you'd like**
I would like to be able to specify the artifact configuration via the environment variables at the pipeline step level ideally (or at least at the pipeline level alternatively).

**Describe alternatives you've considered**
Having one host per pipeline -> does not scale well because we need beefy hosts to test our code (i.e. bare metal instances)
Plumbing this at the environment level could work but it means we need to update this for each newly introduced pipeline, so it is increasing our operational load.

**Additional context**
N/A

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by tracing how artifact configuration is currently supplied through global hooks and environment settings, then determine the pipeline-step configuration boundary. Done means supporting distinct artifact configuration for steps in different pipelines without runtime hook overrides.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ci-cd, devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.