buildkite / buildkite/agent

agent-shutdown hook should trigger on fatal errors

Open
#1,772 0 comments 0 reactions 1 assignee Claimed by @triarius View on GitHub
Dominant language
Go
Stars
1.1k
Forks
378
Avg merge
2d 6h
Merged PRs (30d)
74

Description

**Is your feature request related to a problem? Please describe.**
buildkite workers that encounter network-related errors do not trigger the `agent-shutdown` hook properly.
For example, when the network dies on my agent, and the agent is unable to POST to `https://agent.buildkite.com/v3/register` 30x in a row, the agent dies but does not trigger the `agent-shutdown` hook. I have important cleanup code that must be run after every `buildkite-agent start` command, and so failing to run the `agent-shutdown` hook is a big issue.

**Describe the solution you'd like**
The `agent-shutdown` hook should always be run, as long as the arguments provided to `buildkite-agent` are valid.

**Describe alternatives you've considered**
I can wrap my `buildkite-agent start` invocation in a script that runs the cleanup code there instead, but I feel this rather defeats the purpose of having an `agent-shutdown` hook.

**Additional context**
In our case, we use the `agent-shutdown` hook, combined with `--disconnect-after-job` and to reboot VMs, which are configured to reset to a pristine state on reboot. When the agent fails to connect due internet issues and fails to run the `agent-shutdown` hook, the VMs get stuck. While I will likely implement a fallback rebooting option, it would be nice if `agent-shutdown` was more consistently triggered.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.