Ability to always download artifacts
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
### Proposal
The [artifact](https://www.nomadproject.io/docs/job-specification/artifact) stanza in Nomad doesn't "re-download" the files if the allocation has those files already present. For eg: when an allocation is restarted, the files aren't downloaded again. This makes sense for faster startup times of the task and is a nice feature to have. However, if there's another `always_download=true` kind of flag, that will be super helpful for me.
### Use-cases
I've a Go app that can only start if the file is externally downloaded from an external source like HTTP/S3 etc. The ability to fetch the file is left to external orchestration. This file gets updated at every `n` hours, so my CI system has scheduled pipelines to restart the allocation. However this file doesn't get re-downloaded.
### Attempted Solutions
Since we're migrating from K8s, we had initContainers which did that job for us. But here that isn't possible as noted in https://github.com/hashicorp/nomad/issues/9841#issuecomment-940757058. So now the only real option for me is to use `sidecar=true` and after the file is downloaded, for the shell script to not exit, I've to resort to `sleep infinity` which is undesirable :)
What would be the best way out here? I am happy to send a PR for a new flag `always_download` if that is acceptable. Would just need a basic pointer on where to start looking to change this.
Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the artifact stanza documentation and trace the allocation artifact-download behavior described in the issue. Determine where existing files cause downloads to be skipped and identify the related tests, then define completion as a supported option that fetches artifacts again when an allocation restarts, with coverage for that behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100