Enable Nomad with multi container image sources for high availability
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Proposal
Allow Nomad to have fallback docker image urls to pull from. Lately Gitlab have been somewhat down now and then. Although this is not linked with Nomad itself, it could be an opportunity for improvement. Changing the image config in the docker driver block to become a list instead of a single string so Nomad can try multiple sources from which to pull the container image. Pushing a container to multiple registries is easy to get done in a CI/CD pipeline and is far more convenient than re-evaluating your CI/CD or container registry service provider. (or simply not worth it just for a registry being down or slow occasionally)
Changing:
```
config {
image = "https://hub.docker.internal/redis:7"
}
```
To:
```
config {
image = ["https://hub.docker.internal/redis:7", https://registry.gitlab.com/redis:7", https://gcr.io/redis:7"]
}
```
Could also be:
```
config {
image = ["https://registry1.my.organisation/redis:7", https://registry2.my.organisation/redis:7", https://registry3.my.organisation/redis:7"]
}
```
In the event that the 1st url cannot pull the image, Nomad would automatically try the 2nd url and so on. Would allow Nomad to provide its users with high availability even when the "highly available" service provider is not.
A `retry_pull:` config could also be used in conjunction with this feature to define the number of times to retry pulling from a source before switching to the next.
### Use-cases
- Nomad provides an extra availability layer to protect the user from a crash on the 3rd party side even if it rarely happens.
- Nomad can load balance itself when pulling from a busy registry infrastructure.
- Is aligned with the multi cloud paradigm.
### Attempted Solutions
Could include pulling the image on the target host in the CI/CD pipeline so Nomad does not have to do it.
Could self host a registry but this would add weight on the infrastructure management.
Could consider re-evaluating another service provider but could be incredibly painful or even in some cases just not feasible (Probably the worst solution).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading Nomad's Docker driver configuration and image-pull path, focusing on how the current single image value is handled. Define how multiple registry URLs and optional retry behavior should work, then verify that failed pulls fall back in order without breaking existing single-image configurations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100