ADR: CRD Versioning

Below are some prelimiary thoughts on the topic of CRD versioning, mostly taken from the on-site meeting on this topic. For an ADR, my suggestion is to gather requirements about how it should work from this Kubernetes docs page: Versions in CustomResourceDefinitions - Kubernetes Documentation. It looks like a comprehensive guide on how the versioning works.

Conversion webhooks & up- and downgrading

mutating webhooks are the core mechanism required for versioning CRDs. If an old resource is applied by the user, the webhhook will convert it into the current version. Likewise, if the users requests and older version (arbirary versions can be requested) then the webhook is also used for conversion. (Note from felix: Does that mean that two way conversion is absolutely a thing that needs to be implemented?)

When you read an object, you specify the version as part of the path. You can request an object at any version that is currently served. If you specify a version that is different from the object's stored version, Kubernetes returns the object to you at the version you requested, but the stored object is not changed on disk.

We cannot remove (mandatory) fields, because their content will be required when downgrading. This means we have to rename the fields (i.e. deprecated_oldField).

Only support upgrades for now (no downgrading)
Do not skip releases - always upgrade only one version up (v1 -> v2 -> v3 not v1 -> v3)

CRD size

CRD size is a problem: etcd and the kube API both have limits on how large objects are allowed to be.

copy & paste - To have multiple versions in our CRD, we need to keep the old rust struct around. this means copy pasting the rust struct for each verison. Not ideal ....

stabilize CRDs first to reduce conversion efforts? Would be nice beause it saves a lot of work. But CRD versioning is important now.

ADR thoughts

• What are the decision drivers?
• What could a minimally viable implementation look like?

Start with the linked Kubernetes documentation on Versions in CustomResourceDefinitions, then review the conversion webhook, upgrade-only, release-skipping, and CRD-size considerations in this issue. An ADR is done when it records the decision drivers and a minimally viable implementation, including whether two-way conversion and copied Rust structs are required.