Nomad server leave automatically but not permanently
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Proposal
Have Nomad servers (and potentially clients?) notify the leader whenever they interrupt/terminate. Something like [leave_on_interrupt](https://developer.hashicorp.com/nomad/docs/configuration#leave_on_interrupt) but then less permanent.
For follower-servers, we have to wait a while for the leader to pick up on the follower being offline.
Currently I'm not sure whether leader-servers already do this, but I'd imagine something like `nomad operator step-down` (doesn't exist in Nomad, but does in Vault) would be useful to have Nomad do on shutdown.
In cases where we *know* the server will be offline for a while (e.g. reboot), there's no need to wait for timeouts/heartbeats.
### Use-cases
- Checking server-health more reliably (e.g. when deciding "Can I, a follower node, reboot right now?`)
### Attempted Solutions
- Waiting about 5-10 seconds to see `Alive` change in `Left`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the existing leave_on_interrupt behavior and the Nomad server membership states mentioned in the issue, including Alive and Left. Compare the requested behavior with the nomad operator step-down precedent, then clarify whether the scope includes follower servers, leader servers, or clients; done requires an agreed shutdown-notification design that avoids waiting for timeout or heartbeat detection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100