buildkite / buildkite/agent-stack-k8s
Only set necessary environment variables on system containers
- Dominant language
- Go
- Stars
- 110
- Forks
- 57
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 5
Description
[Right now](https://github.com/buildkite/agent-stack-k8s/blob/59594d25bc830fa3b67d503dd55dadec663221f6/scheduler/scheduler.go#L349) we're having the `checkout` container copy the `~/.ssh` directory and set permissions on it, taking advantage of the fact that it previously didn't have a `command` phase.
The problem is that a common pattern for when we're using Alpine Linux containers is[ to set](https://github.com/buildkite/agent-stack-k8s/blob/fd9da3fce33c63b12c635d3537564f9ec5c84a64/.buildkite/pipeline.yml#L78) `BUILDKITE_SHELL` to `/bin/sh`, because Alpine doesn't come out of the box with bash. With the way that the environment gets set in each of these containers, this means that `BUILDKITE_SHELL` then gets set to that on the checkout container, resulting in this weird error of
```
/bin/sh: can't open 'trap 'kill -- $' INT TERM QUIT; cp -r ~/.ssh /workspace/.ssh && chmod -R 777 /workspace': No such file or directory
```
One way of solving this would be to just not set `BUILDKITE_SHELL` on the checkout container, we already do some kind of deny-listing like this [with other parts of the environment](https://github.com/buildkite/agent-stack-k8s/blob/59594d25bc830fa3b67d503dd55dadec663221f6/scheduler/scheduler.go#L215).
Another way would be to find the minimum subset of variables that the system containers need (agent, checkout, artifact upload), and only give the entire environment to the command/sidecar containers.
A third way we could solve this would be to move this logic into the agent itself, rather than using the command step like this.
A way that users can work around this is to [set the env only on the specific Alpine container](https://github.com/buildkite/agent-stack-k8s/blob/59594d25bc830fa3b67d503dd55dadec663221f6/.buildkite/pipeline.yml#L87-L89), but I'm thinking that the current default when you set `env` at the step level might be surprising behavior for some.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.