carvel-dev / carvel-dev/imgpkg
Pulling a nested bundle should allow filtering
- Dominant language
- Go
- Stars
- 284
- Forks
- 70
- PR merge metrics
- No merged PRs in 30d
Description
# Use Case
As an Application consumer
When consuming a Bundle that contains *lots* of Nested Bundles
I want control over which Nested Bundles are pulled
So that less compute and storage resources are consumed
And to reduce the time it takes to pull Nested Bundles
# Description
Introduce a new flag `--recursive-filter` to the `pull` command, accepting multiple filters to apply when pulling a Bundle.
A Filter is a key-value that matches against an Images annotation contained in a Bundle's `ImageLock` file. If the image's annotation matches the provided filter/s then the image (which will always be a bundle) is fetched. Otherwise it is skipped.
# Acceptance Criteria
Given the following Bundle with Nested Bundle relationship:
```
my.registry.io/bundle-1 -> my.registry.io/bundle-2 -> my.registry.io/bundle-3
-> my.registry.io/ubuntu
```
And
The following bundles are structured:
- `my.registry.io/bundle-1@sha256:123`
The `.imgpkg/images.yml` looks like the following
```yaml
apiVersion: imgpkg.carvel.dev/v1alpha1
kind: ImagesLock
images:
- image: my.registry.io/bundle-2@sha256:456
annotations:
BUNDLE-2-ANNOTATION: "true"
- image: my.registry.io/ubuntu@sha256:999
annotations:
IMAGE-UBUNTU-ANNOTATION: "true"
```
- `my.registry.io/bundle-2@sha256:456`
The `.imgpkg/images.yml` looks like the following
```yaml
apiVersion: imgpkg.carvel.dev/v1alpha1
kind: ImagesLock
images:
- image: my.registry.io/bundle-3@sha256:789
annotations:
BUNDLE-3-ANNOTATION: "true"
```
The following scenarios should be fulfilled:
🟢 Scenario 0: Every image referenced by the 'root' Bundle is fetched
```
imgpkg pull -b my.registry.io/bundle-1 --recursive
```
Results in every bundle being fetched.
Expected Standard Output:
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)
Pulling Nested Bundle 'my.registry.io/bundle-2@sha256:456'
Extracting layer 'sha256:111' (1/1)
Pulling Nested Bundle 'my.registry.io/bundle-3@sha256:789'
Extracting layer 'sha256:111' (1/1)
```
🟢 Scenario 1: Only first level Nested Bundle is fetched
```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter BUNDLE-2-ANNOTATION=true
```
Results in only bundle-2 image being fetched.
Expected standard output
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)
Pulling Nested Bundle 'my.registry.io/bundle-2@sha256:456'
Extracting layer 'sha256:111' (1/1)
Skipped Pulling Nested Bundle 'my.registry.io/bundle-3@sha256:789' (due to recursive-filter flag)
```
🟢 Scenario 2: Every Nested Bundle is fetched
```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter BUNDLE-2-ANNOTATION=true --recursive-filter BUNDLE-3-ANNOTATION=true
```
Results in every bundle being fetched.
Expected Standard Output:
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)
Pulling Nested Bundle 'my.registry.io/bundle-2@sha256:456'
Extracting layer 'sha256:111' (1/1)
Pulling Nested Bundle 'my.registry.io/bundle-3@sha256:789'
Extracting layer 'sha256:111' (1/1)
```
🟢 Scenario 3: Every Nested Bundle is skipped
```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter BUNDLE-3-ANNOTATION=true
```
Results in every nested bundle skipped. This is because the intermediate Bundle (Bundle-2) was not fetched. And thus, the `ImagesLock` file contained in Bundle-2 was not read / processed.
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)
Skipped Pulling Nested Bundle 'my.registry.io/bundle-2@sha256:456' (due to recursive-filter flag)
```
🟢 Scenario 4: Filter flag is used, but the recursive flag is not provided
```
imgpkg pull -b my.registry.io/bundle-1 --recursive-filter BUNDLE-2-ANNOTATION=true
```
Results in only the root bundle 'bundle-1' being fetched.
Expected standard output
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)
```
🔴 Scenario 5: Filter flag is incorrectly used
```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter
```
or
```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter ONLY_KEY_PROVIDED
```
Results in imgpkg `pull` command error-ing out.
Example of a possible standard Error message given to the user
```
imgpkg pull error: --recursive-filter applies against keys and values. Only the key was provided.
```
### Notes:
- Filtering on key-values *are* case sensitive.
### Out of scope:
- wildcard value matching
- defaulting an empty value in the filter e.g. `--recursive-filter annotation_key=`
- UX warning messages around using `--recursive-filter` and not providing `--recursive` flag. See: https://github.com/vmware-tanzu/carvel-imgpkg/issues/118
Contributor guide
Assessment
This issue has not been assessed yet.