Rework container image rebuilds
- Dominant language
- C#
- Stars
- 181
- Forks
- 67
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 10
Description
## Current state
```mermaid
graph TD
subgraph docker-tools repo
check-base-images["check base images pipeline \n (multiple repos)"]
subscriptions[subscriptions.json]
end
subgraph consuming["consuming repo"]
build-pipeline[build-pipeline]
end
subgraph versions["versions repo"]
image-info[image-info.json]
end
subgraph acr
public["public images"]
end
check-base-images -->|reads| subscriptions
check-base-images -->|"reads (many)"| image-info
check-base-images -->|"queues (many)"| build-pipeline
subscriptions -->|"references (many)"| consuming
build-pipeline <-->|updates and reads| image-info
build-pipeline -->|pushes images| public
```
### Problems with current setup
- Requires a separate repo to store `image-info.json` metadata file. Git is in the critical path for builds.
- Only the docker-tools repo can queue base image re-bulids, and the docker-tools repo must know about all other consuming repos in order to queue their builds.
- The docker-tools repo and the consuming repos can go out-of-sync. For example, if the manifest.json or image-info formats change, then the docker-tools check-base-images pipeline can fail until all consuming repos have adopted the updated image-builder version.
## Desired state
**Key ideas**:
- Consuming repos should own their own base image update pipeline + schedule.
- The docker-tools repo should provide sane defaults/pipeline templates for base image re-builds.
- Base image re-builds should work both in public and internal scenarios without extra work.
- There should not be a dependency on external repos or storage locations for the critical base image re-build path. `image-info.json`, if it still needs to exist, should not be a source of truth. It should be computed separately, strictly downstream of image build pipelines.
```mermaid
graph TD
subgraph consuming[consuming repo]
check-base-images["check base images pipeline \n (one repo)"]
build-pipeline[build pipeline]
end
subgraph acr
public["public images \n with labels/annotations"]
end
public -->|reads| check-base-images
public -->|reads| build-pipeline
check-base-images -->|queues| build-pipeline
build-pipeline -->|pushes images| public
```
## Future work
Here's the work that I see required to reach this desired state:
- [ ] #1602
- [ ] Attach metadata labels to images during build - #2163
- [ ] Make rebuild decisions repo-local
- [ ] Implement a new version of the `getStaleImages` command which reads these labels to determine which images are out of date.
- [ ] Implement a new pipeline which runs the new command and queues a build for stale images.
- [ ] Backwards compatibility with existing image-info.json publishing
- ImageBuilder should still synthesize image-info.json and write it to disk, at least for the time being.
- For backwards compatibility, we need to provide some kind paved path for publishing image-info.json to a location like a git repo, OCI artifact, or storage account. This could be in the form of a pipeline extension point/parameter, a new imagebuilder command, a totally new pipeline/template, or some combination of those.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.