bazel-contrib / bazel-contrib/rules_oci
FR: Shallow pull for oci_pull
- Dominant language
- Starlark
- Stars
- 423
- Forks
- 213
- Avg merge
- 53m
- Merged PRs (30d)
- 1
Description
Currently, base image is input to oci_image. This has some downsides
- Pushing giant blobs to remote action cache is slow and takes more time than building the target from scratch
- Copying blobs from base image into the oci_image output takes time (see #365)
- Downloading the base image from the internet takes time
I am proposing a shallow pull feature that will prevent this from happening by only downloading necessary image metadata such as `manifest` and `config` so we can mutate the config and the manifest. We never mutate existing layers so this is safe.
However, it's not as easy it sounds since we have to
- instruct `crane` to be happy with missing blobs on disk (to be done)
- oci_tarball has to be aware of shallow oci-layouts and create necessary metadata on its tarball `manifest` so the runtime knows where to get the blobs from
- oci_push needs to know how to deal oci-layouts, eg knowing how to `mount` blobs (see https://github.com/google/go-containerregistry/issues/1321). Feature is already there but we might need to change oci_push a little bit for this to work.
We won't be offering this feature by default as we can't be certain every tooling will know how to deal with shallow oci-layouts.
Contributor guide
Research direction
Start with the oci_pull, oci_tarball, and oci_push entry points named in the issue, then investigate how crane handles missing blobs. Check the linked go-containerregistry issue for blob mounting context. Done means shallow metadata-only pulls work through tarball creation and pushing without requiring existing layers locally.
Written by the indexing model from the issue text.
Assessment
- Domain
- build-system, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100