[Feature Request] KaaS - CRUD API for Clusters

Abstract

For a new offering at Cloudogu we have implemented dynamic provisioning of K8s clusters against the APIs of different cloud providers to make good on our promise of vendor independence.

Implementing this per provider is tedious. I think this is where the SCS KaaS standard could shine!
Couldn't there be a standardized API to do this vendor-independently?
For us, as a consumer, this would provide the real value of a sovereign cloud standard: Implement once, support multiple SCS KaaS-compliant providers.

Initial thoughts

I am new to the SCS community and no expert on the provider side, more like an experienced consumer of Kubernetes clusters.
To get the discussion started, I spent some time analyzing the current situation and came up with some questions that I like to hear your thoughts on.

The biggest challenge I see here is different implementations/APIs among the standardized KaaS providers.
Looking at the existing SCS KaaS-certified products from noris, ScaleUp and Syself, I can see three different implementations: Gardener, ClusterAPI, OpenStack.

From a consumer's point of view, it would be great to be able to choose between different clients like Terraform/OpenTofu, CLI, generic HTTP and (depending on the API) Kubernetes CRDs.

I created a summary of what the different implementations offer in this regard:

Implementation	Terraform/OpenTofu	CLI	HTTP	CRD
OpenStack	openstack	openstack	openstack API	-
CAPI	kubectl/Kubernetes	kubectl	Kubernetes API	Cluster
Gardener	gardener	kubectl / gardenctl	Kubernetes API	Shoot

Can one of these be chosen as the standard KaaS API (like OpenStack in SCS IaaS)?

I can imagine that it will be a challenge to chose one, especially so because they have different attributes (like Kubernetes-native CRDs) and tools involved and would force providers to build adapters. I don't know if this is realistic.

CAPI and Gardener as Kubernetes-native implementations seem closer to each oter than OpenStack.
With a tool like cluster-api-provider-gardener (CAPGa) (unstable, unfortunately)
would we be able to agree on CAPI as the standard?

For OpenStack-based products it would probably be challenging to support CRDs, right?

So maybe the solution here could be that providers either offer an OpenStack API or one Kubernetes-native API?

Creating a new SCS API would cause a lot of effort for all providers and might hinder adoption, so I don't think this would be wise.

What is the most pragmatic solution here?

Stakeholders

In an initial talk with @depressiveRobot I got the impression that he considers this a sensible enhancement for a future version of SCS KaaS.

First, I would like to add the representatives of SCS-compatible KaaS products to the discussion:

• noris Sovereign Cloud (nSC): @benedikt-haug
• ScaleUp Open Cloud: @fzakfeld
• Syself Autopilot: @janiskemper

In addition, I know that Cloud & Heat are considering a KaaS certification, so @horazont @Felkonsky might be interested in this discussion.

At SCS Summit, @majst01 introduced metal-stack.io. I can imagine metal-stack could also bring a lot of experience into this discussion.

Please add all others that might be interested in joining this effort.

Closing thoughts

I'd be willing to join SIG standardization meetings to discuss in person. Just let me know which meeting will be a good fit.

In addition, I can offer to evaluate any POC/MVP of a standardized SCS KaaS CRUD API with different providers and provide feedback and a real-world use-case.

This is my first issue with SCS. Looking forward to working with you all!

Start with the issue's comparison of OpenStack, CAPI, and Gardener implementations and the listed SCS KaaS stakeholders. No files, tests, or implementation entry points are identified; done would require agreement on a pragmatic vendor-independent CRUD API approach and its scope for a future KaaS standard.