Pre-pull curated images onto every node
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Task Summary
The first unit on a node waits for the whole image — about 80 seconds for a 3 GB one — while every later unit there starts at once. The same action takes seconds or minutes depending on which node it lands on.
Pull each ready image onto every node as soon as it is ready, rather than when someone first starts a unit from it.
Same mechanism Texera already uses for the default computing-unit image: a DaemonSet whose init container is the image and whose command does nothing, then a small pause container to hold the pod open so the image is not reclaimed. One per image, removed when the image is.
If it cannot start it is logged and ignored — the image still works, the first unit just pays for the pull.
Costs node disk: every node holds every ready image. Configurable.
Done when a ready image is on every node before anyone starts a unit from it, and deleting the image removes its pre-pull.
Part of #8466
### Task Type
- [ ] Refactor / Cleanup
- [ ] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other
Contributor guide
Assessment
This issue has not been assessed yet.