progress_deadline distinguish between updates (auto_revert) and deployments
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Proposal
Add a new progress_deadline setting that only applies to updates and auto_revert decision making.
Today if you set progress_deadline to 5m and deploy a change to a job that, lets say, references a non-existing docker image, and set auto_revert to true; it will try for 5 minutes, fail (image not found), then revert to the old deployment.
This sounds good.
But what if, lets say the docker-registry is down for 15 minutes and you try to deploy a change?
I believe it will try for five minutes, fail, revert, continue to fail for five minutes (the problem wasnt the job, it was the docker-registry), and then give up...forever
Nomad giving up forever is a very bad and unexpected state. To get around it, maybe we want to set progress_deadline to 99999h (wish we could set unlimited = true here), but then you break the first case - auto_revert can never work.
So what I think I want is to tell nomad that if you are deciding whether to revert or not, consider the deadline 5m
But if you are deciding whether to fail the deployment, and wont revert, but will leave the job dead, consider a different deadline.
Why is the deadline for reverting the same as leaving the job dead forever?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the deployment behavior described in the issue for progress_deadline and auto_revert. Compare the non-existent image case with the registry outage case, then define separate deadline behavior that preserves auto-revert while avoiding a permanent failure state. Done means the semantics are agreed and covered by tests for both scenarios.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100