rolling upgrades / gradual rollout for batch type jobs
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Nomad version
- Nomad v0.12.9
- Nomad v1.0.1
### Issue
It appears there is no way to use "update" stanza on "batch" type job to stagger. Only on "service" and "system" type jobs.
In other words, [rolling upgrades](https://learn.hashicorp.com/tutorials/nomad/job-rolling-update) seem to be unsupported for "batch" type jobs.
Unfortunately, "service" type job isn't flexible enough for my use case, and using "system" job feels wrong.
Executed script could be updated to use a locking mechanism, making it wait until the lock is available before proceeding. But that feels like over engineering the process.
Is there some other Nomad way to achieve a gradual rollout / execution of "batch" job?
Use case: using `raw_exec` to execute script performing complex deployment on hosts gradually, ensuring such execution will not cause an outage by making application unavailable on all targeted hosts.
/cc @tgross
### Reproduction steps
- save job file example as `batch-test.nomad`
- run `nomad job validate batch-test.nomad`
### Job file
```
job "deploy-MyApp-297209" {
datacenters = ["DC1"]
type = "batch"
# make sure to run on windows hosts only
constraint {
attribute = "${attr.kernel.name}"
value = "windows"
}
# run on hosts designated for specific application
constraint {
attribute = "${meta.application}"
operator = "regexp"
value = "(?i)MyApp"
}
# do not run more than one on the same node
constraint {
distinct_hosts = true
}
# rolling updates, run one server at a time
update {
max_parallel = 1
stagger = "60s"
}
group "deploy" {
count = 4
# retry failed jobs
restart {
attempts = 3
delay = "10s"
mode = "fail"
}
# updates shared scripts
task "update_scripts" {
driver = "raw_exec"
config {
command = "powershell"
args = [
"-command",
<<-EOS
.. do prestart things ..
EOS
]
}
lifecycle {
hook = "prestart"
sidecar = false
}
}
# runs commands to copy site from artifact destination
# directory to permanent site's root and performs deployment tasks
task "deploy_artifacts" {
driver = "raw_exec"
# site artifact
artifact {
source = ".. some source .."
destination = "local/"
}
config {
command = "powershell"
args = [
"-command",
<<-EOS
.. do deploy things ..
EOS
]
}
}
}
}
```
### Nomad Client logs
```sh
$ nomad job validate batch-test.nomad
Job validation errors:
1 error occurred:
* Task group deploy validation failed: 1 error occurred:
* Job type "batch" does not allow update block
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.