carvel-dev / carvel-dev/imgpkg

Pulling a nested bundle should allow filtering

Open
#116 1 comment 0 reactions 0 assignees View on GitHub
carvel accepted priority/unprioritized-backlog
Dominant language
Go
Stars
284
Forks
70
PR merge metrics
No merged PRs in 30d

Description

# Use Case
As an Application consumer
When consuming a Bundle that contains *lots* of Nested Bundles
I want control over which Nested Bundles are pulled
So that less compute and storage resources are consumed
And to reduce the time it takes to pull Nested Bundles

# Description

Introduce a new flag `--recursive-filter` to the `pull` command, accepting multiple filters to apply when pulling a Bundle.

A Filter is a key-value that matches against an Images annotation contained in a Bundle's `ImageLock` file. If the image's annotation matches the provided filter/s then the image (which will always be a bundle) is fetched. Otherwise it is skipped.

# Acceptance Criteria

Given the following Bundle with Nested Bundle relationship:
```
my.registry.io/bundle-1 -> my.registry.io/bundle-2 -> my.registry.io/bundle-3
-> my.registry.io/ubuntu
```

And

The following bundles are structured:
- `my.registry.io/bundle-1@sha256:123`
The `.imgpkg/images.yml` looks like the following
```yaml
apiVersion: imgpkg.carvel.dev/v1alpha1
kind: ImagesLock
images:
- image: my.registry.io/bundle-2@sha256:456
annotations:
BUNDLE-2-ANNOTATION: "true"
- image: my.registry.io/ubuntu@sha256:999
annotations:
IMAGE-UBUNTU-ANNOTATION: "true"
```

- `my.registry.io/bundle-2@sha256:456`
The `.imgpkg/images.yml` looks like the following
```yaml
apiVersion: imgpkg.carvel.dev/v1alpha1
kind: ImagesLock
images:
- image: my.registry.io/bundle-3@sha256:789
annotations:
BUNDLE-3-ANNOTATION: "true"
```

The following scenarios should be fulfilled:
🟢 Scenario 0: Every image referenced by the 'root' Bundle is fetched
```
imgpkg pull -b my.registry.io/bundle-1 --recursive
```

Results in every bundle being fetched.

Expected Standard Output:
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)

Pulling Nested Bundle 'my.registry.io/bundle-2@sha256:456'
Extracting layer 'sha256:111' (1/1)

Pulling Nested Bundle 'my.registry.io/bundle-3@sha256:789'
Extracting layer 'sha256:111' (1/1)
```

🟢 Scenario 1: Only first level Nested Bundle is fetched

```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter BUNDLE-2-ANNOTATION=true
```

Results in only bundle-2 image being fetched.

Expected standard output
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)

Pulling Nested Bundle 'my.registry.io/bundle-2@sha256:456'
Extracting layer 'sha256:111' (1/1)

Skipped Pulling Nested Bundle 'my.registry.io/bundle-3@sha256:789' (due to recursive-filter flag)

```

🟢 Scenario 2: Every Nested Bundle is fetched
```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter BUNDLE-2-ANNOTATION=true --recursive-filter BUNDLE-3-ANNOTATION=true
```

Results in every bundle being fetched.

Expected Standard Output:
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)

Pulling Nested Bundle 'my.registry.io/bundle-2@sha256:456'
Extracting layer 'sha256:111' (1/1)

Pulling Nested Bundle 'my.registry.io/bundle-3@sha256:789'
Extracting layer 'sha256:111' (1/1)
```

🟢 Scenario 3: Every Nested Bundle is skipped
```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter BUNDLE-3-ANNOTATION=true
```

Results in every nested bundle skipped. This is because the intermediate Bundle (Bundle-2) was not fetched. And thus, the `ImagesLock` file contained in Bundle-2 was not read / processed.

```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)

Skipped Pulling Nested Bundle 'my.registry.io/bundle-2@sha256:456' (due to recursive-filter flag)
```

🟢 Scenario 4: Filter flag is used, but the recursive flag is not provided

```
imgpkg pull -b my.registry.io/bundle-1 --recursive-filter BUNDLE-2-ANNOTATION=true
```

Results in only the root bundle 'bundle-1' being fetched.

Expected standard output
```
Pulling bundle 'my.registry.io/bundle-1'
Extracting layer 'sha256:111' (1/1)
```

🔴 Scenario 5: Filter flag is incorrectly used

```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter
```

or

```
imgpkg pull -b my.registry.io/bundle-1 --recursive --recursive-filter ONLY_KEY_PROVIDED
```

Results in imgpkg `pull` command error-ing out.

Example of a possible standard Error message given to the user

```
imgpkg pull error: --recursive-filter applies against keys and values. Only the key was provided.
```

### Notes:

- Filtering on key-values *are* case sensitive.

### Out of scope:
- wildcard value matching
- defaulting an empty value in the filter e.g. `--recursive-filter annotation_key=`
- UX warning messages around using `--recursive-filter` and not providing `--recursive` flag. See: https://github.com/vmware-tanzu/carvel-imgpkg/issues/118

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.