devopshq / devopshq/artifactory-cleanup

ExcludeDockerImages masks end up in missing layers

Open
#121 7 comments 1 reaction 0 assignees View on GitHub
bug good first issue help wanted
Dominant language
Python
Stars
160
Forks
87
PR merge metrics
No merged PRs in 30d

Description

this is the used config:

```
- name: Remove all docker images from docker-xyz-local older than 14 days
rules:
- rule: Repo
name: docker-xyz-local
- rule: DeleteDockerImagesOlderThan
days: 14
- rule: ExcludeDockerImages
masks:
- image/path/with:tag
```

I expected the image `image/path/with:tag` to be excluded from deletion, with all its layers. Is that an wrong assumption with this config?

`list.manifest.json` of this image (note the digest `877fe4031a674aebb3d006692366fb73badfc36e0d1756964b9d1a7db5b8b553`):

```
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:a7660fa61e8845d998169dc6c6a2cf801284af6a055ea52b94aad540c68d8926",
"size": 2199,
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:877fe4031a674aebb3d006692366fb73badfc36e0d1756964b9d1a7db5b8b553",
"size": 566,
"annotations": {
"vnd.docker.reference.digest": "sha256:a7660fa61e8845d998169dc6c6a2cf801284af6a055ea52b94aad540c68d8926",
"vnd.docker.reference.type": "attestation-manifest"
},
"platform": {
"architecture": "unknown",
"os": "unknown"
}
}
]
}

```

the output of the deletion shows the `nmatch` entry for the image, but also the deletion of its layer:

```
Add AQL Text - rule: Repo - Apply the policy to one repository.
Add AQL Text - rule: DeleteDockerImagesOlderThan - Removes Docker image older than ``days`` days
Add AQL Text - rule: ExcludeDockerImages - Exclude Docker images by name and tags.
********************************************************************************
Result AQL Query:

items.find(
...
{"path": {"$nmatch": "image/path/with/tag"}}

Found 1093 artifacts
Filter artifacts - rule: Repo - Apply the policy to one repository.
Filter artifacts - rule: DeleteDockerImagesOlderThan - Removes Docker image older than ``days`` days
Filter artifacts - rule: ExcludeDockerImages - Exclude Docker images by name and tags.
Found 1093 artifacts AFTER filtering

DESTROY MODE - delete 'docker-xyz-local/image/path/sha256%3A877fe4031a674aebb3d006692366fb73badfc36e0d1756964b9d1a7db5b8b553 - 1K'
...

```

After that, the image can't be pulled any more, as the layer is missing in Artifactory. This doesn't happen all the time, only for some images or single layers. How can I avoid this?

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the configured Repo, DeleteDockerImagesOlderThan, and ExcludeDockerImages rules with the provided image path and manifest digest. Start by tracing how the generated $nmatch result is applied to manifest and layer artifacts, then compare the deletion output with the manifest references. Done means the excluded image remains pullable and its referenced layers are not deleted.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.