carvel-dev / carvel-dev/kapp

orphan delete-strategy for child resources not created by kapp

Open
#215 3 comments 0 reactions 0 assignees View on GitHub
helping with an issue
Dominant language
Go
Stars
1.1k
Forks
134
PR merge metrics
No merged PRs in 30d

Description

I have an app that I deploy and teardown all the time as part of an automated test process. Part of my app includes certificates generated by cert-manager and provisioned by Let's Encrypt.

Up until now, I excluded the certificate resources from the automated test environment because the act of creating and deleting the certificate resources leads to exceeding Let's Encrypt's [rate limits](https://letsencrypt.org/docs/rate-limits/), which causes my deployments to stop working.

I noticed kapp features a `kapp.k14s.io/delete-strategy` annotation that I might be able to use on the certificate to "skip deleting" it and reuse it on the next deployment. I thought this would be better for me because having the certificates in my test environment just like every other environment would make my test environment more production-like and simplify my test process by letting it assume there will always be valid certificates in any environment.

I ran into issues with the generated, "child resources" of the certificate.

**What steps did you take:**

I applied the `kapp.k14s.io/delete-strategy: orphan` annotation to my `certificate.cert-manager.io` resource.
I deployed the app.
I deleted the app.

**What happened:**

When the app deployed, cert-manager created the accompanying `certificaterequest.cert-manager.io` and `order.acme.cert-manager.io` resources (normal, desired behavior of cert-manager). A `kubectl describe` command for each of the generated resources shows that they have inherited the `kapp.k14s.io/delete-strategy: orphan` annotation. I don't know what mechanism applied that annotation to them (maybe cert-manger or k8s itself?), but that's exactly what I wanted so I'm happy about it. The three resources work as a unit so I wanted to treat all of the same way (i.e. leave them orphaned on delete).

```
Name: my-app-qcmbr
Namespace: my-app
Labels: kapp.k14s.io/app=1619283995411166000
kapp.k14s.io/association=v1.8dd04f03c81055b2dff27e72ffe84719
Annotations: cert-manager.io/certificate-name: my-app
cert-manager.io/certificate-revision: 1
cert-manager.io/private-key-secret-name: my-app-tjh4z
kapp.k14s.io/delete-strategy: orphan 👈 😁
kapp.k14s.io/identity: v1;my-app/cert-manager.io/Certificate/my-app;cert-manager.io/v1
kapp.k14s.io/original:
{"apiVersion":"cert-manager.io/v1","kind":"Certificate","metadata":{"annotations":{"kapp.k14s.io/delete-strategy":"orphan"},"labels":{"kap...
kapp.k14s.io/original-diff-md5: 58e0494c51d30eb3494f7c9198986bb9
API Version: cert-manager.io/v1
Kind: CertificateRequest
```

However when I deleted the app, kapp's diff didn't show the orphan strategy for the generated child resources.

```
Namespace Name Kind Conds. Age Op Op st. Wait to Rs Ri
(cluster) my-app Namespace - 9m delete orphan delete ok -
my-app api Deployment 2/2 t 19s delete - delete ok -

... (abbreviated)

^ my-app Certificate 1/1 t 9m delete orphan delete ok -
^ my-app-qcmbr CertificateRequest 1/1 t 9m - - delete ok - 👈 🤔
^ my-app-qcmbr-1942737023 Order - 9m - - delete ok - 👈 🤔
```

Oh no! This diff from kapp isn't representing `kapp.k14s.io/delete-strategy: orphan` annotations on CertificateRequest and Order resources! I went ahead and deleted the app anyway to see what would happen.

```
$ kapp -n apps delete -a my-app
... (abbreviated)
10:17:51AM: ---- waiting complete [28/28 done] ----

kapp: Error: Expected all resources to be gone, but found: order/my-app-qcmbr-1942737023 (acme.cert-manager.io/v1) namespace: my-app, certificaterequest/my-app-qcmbr (cert-manager.io/v1) namespace: my-app
$ echo $?
1
```

So, it seems it didn't delete the resources (yay) but printed a sad error message and exited unsuccessfully (boo).

**What did you expect:**

- show the delete strategy on the child resources in kapp's diff
- `kapp delete` doesn't delete the child resources (current behavior, I guess)
- `kapp delete` exists successfully

**Anything else you would like to add:**

The Carvel team rocks.

**Environment:**

- kapp version (use `kapp --version`): 0.36.0
- OS (e.g. from `/etc/os-release`): Alpine and macOS
- Kubernetes version (use `kubectl version`): 1.19.7

---
Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.