hashicorp / hashicorp/nomad

(pre-)Stop/Kill action/command

Open
#9,872 12 comments 31 reactions 0 assignees View on GitHub
hcc/jira stage/needs-discussion theme/task lifecycle type/enhancement
Dominant language
Go
Stars
17k
Forks
2.1k
Avg merge
1d 9h
Merged PRs (30d)
105

Description

Currently Nomad supports defining a [kill signal](https://www.nomadproject.io/docs/job-specification/task#kill_signal), and it'd be pretty useful to be able to define pre- and stop/kill actions/commands ( we can already do post-stop via tasks with lifecycle > hook > poststop).

The main use case i see for this is shutting down complex software/tasks that needs actions performed on it for a graceful shutdown, e.g. [ScyllaDB](https://docs.scylladb.com/operating-scylla/procedures/cluster-management/safe-shutdown/) recommend running a command (`nodetool drain`) and then shutting down gracefully before killing the Docker container.

It could also be useful in order to do more graceful drains, for example when doing rolling upgrades (e.g. failing the healthcheck to make the instance inaccessible from Consul/LB before actually shutting it down).

In theory it could be achieved with an additional hook (prestop), but that might cause some issues ( e.g. in ScyllaDB's case, the prestop task would need to contain all the tools and configuration to be able to run commands on the ScyllaDB running in the main task; and it won't work for the specific case, since they recommend shutting down gracefully via supervisord after draining, and i don't think one can call supervisorctl remotely).

Adrian

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing Nomad's existing kill_signal behavior and the poststop task lifecycle described in the issue. Define the expected semantics for pre-stop and stop/kill actions, including graceful shutdown and rolling-upgrade draining, then verify the behavior against the ScyllaDB use case.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.