[Sandbox] KubeVM and vSphere Provider for KubeVM

### Project summary

KubeVM is a vendor-neutral Kubernetes API for managing full lifecycle of virtual machines on any hypervisor, implemented through pluggable providers.

### Project description

**What does KubeVM do?**

KubeVM introduces a portable `VirtualMachine` custom resource in `kube-vm.io` API group that represents a hypervisor agnostic virtual machine on a Kubernetes cluster. It also introduces a set of controllers to reconcile these resources. The exact provisioning and management of virtual machines is delegated via a pluggable provider implementation to the infrastructure providers so they can leverage the true potential of their platform.

The project also introduces primitives on top of virtual machines such as `VirtualMachineReplicaSets`, `VirtualMachineDeployments` that can be used to dynamically scale up workloads to achieve a desired set of replicas along with a versioning and an upgrade strategy.

This project also provides a set of APIs that enable consistent experience for common set of operations on virtual machines regardless of the underlying hypervisor - such as bootstrapping (using [cloud-init](https://cloud-init.io/), [Sysprep](https://en.wikipedia.org/wiki/Sysprep)), Snapshots, Recovery etc.

KubeVM builds on the solid foundation of [vm-operator](https://github.com/vmware-tanzu/vm-operator/) (described below) which is an open-source project consumed by [Cluster API Provider for vSphere](https://github.com/kubernetes-sigs/cluster-api-provider-vsphere/) serving as a foundation for [VKS](https://www.vmware.com/products/cloud-infrastructure/vsphere-kubernetes-service) (VMware vSphere Kubernetes Service).

The `VirtualMachine` API introduced by KubeVM:
* Carries the intent common to every platform: power state, sizing, boot image, storage, networking, guest bootstrap.
* It binds to a platform through a reference to a provider-owned object that carries only what is genuinely platform-specific.
* The generic controller observes the backend through a duck-typed status contract of well-known field paths, so it imports no provider code and each provider evolves independently behind that contract.

The pattern is Cluster API's core-plus-providers split, applied to VM lifecycle rather than cluster lifecycle.

**Core capabilities of the API**

* **Declarative VM lifecycle**: create, power on/off/suspend, delete, with power state modelled as desired state and destruction as object deletion.
* **Flexible virtual hardware specification**: sizing by reference to an administrator-published profile, in the manner of cloud instance types, or by explicit requests for CPU and memory.
* **Portable boot and storage**: image and snapshot references, additional disks, and disk performance expressed by referencing a `StorageClass` and `VolumeAttributesClass` so the existing CSI ecosystem remains unchanged.
* **Networking**: multiple interfaces, static or dynamic addressing, guest-wide settings, external address requests.
* **Guest bootstrap**: cloud-init supplied through Secret references.
* **Provider-agnostic status**: a fixed status contract plus a condition vocabulary that reports capability gaps, pending changes, and retryable platform failures rather than failing writes.

**Why is KubeVM needed?**

Kubernetes has become the default control plane for infrastructure, but there is no vendor-neutral, VM-centric API through which platforms can expose virtual machines. The consequences are concrete:

* A VM manifest is written against one platform's controller, so multi-platform estates maintain a separate manifest and a separate operational model per platform.
* Higher-level orchestration such as ReplicaSets / Deployments, load balancers have to be rebuilt per platform, because there is no portable machine object to build it on.
* Organizations with an existing virtualization estate have no Kubernetes-native front door to it that preserves that platform's own live migration, storage, and high availability.

KubeVM addresses these directly, and the value compounds: once a machine is portable, controllers layered above it are written once and work on every provider. It is the same leverage `StatefulSets/Deployments` and `Services` gained from `Pods` being a stable contract.

**Current state**

The proposal contains a first version of the [proposed API types](https://github.com/vmware-tanzu/vm-operator/blob/feature/kube-vm/external/kubevm/api/v1alpha1/virtualmachine_types.go). It also includes the [CRD](https://github.com/vmware-tanzu/vm-operator/blob/feature/kube-vm/external/kubevm/config/crd/bases/kube-vm.io_virtualmachines.yaml) generated from the types which have been installed against a live Kubernetes API server, and validated to yield real objects. The generic controller is the first milestone on the roadmap and is not yet written. The design is open for public comment as https://github.com/vmware-tanzu/vm-operator/pull/1893.

[VM Operator](https://github.com/vmware-tanzu/vm-operator) is the first provider implementation. It is a production Kubernetes-native VM controller validated at 25,000 VMs per cluster, shipping today as the control plane for VMware's VM Service.

### Project vs Reference Architecture/Implementation

- [x] This is a reusable open source project designed for broad adoption, not a reference architecture, reference implementation, or demonstration of patterns

### Org repo URL (provide if all repos under the org are in scope of the application)

N/A

### Project repo URL in scope of application

https://github.com/vmware-tanzu/vm-operator/

### Additional repos in scope of the application

KubeVM code currently lives in a managed, long running [feature/kube-vm](https://github.com/vmware-tanzu/vm-operator/tree/feature/kube-vm) branch of VM Operator. The [proposal](https://github.com/vmware-tanzu/vm-operator/pull/1893) for the API is being reviewed by the community. We welcome contributions and feedback from the community to this branch.

The [external/kubevm](https://github.com/vmware-tanzu/vm-operator/tree/feature/kube-vm/external/kubevm) directory from this branch will be spun out as the KubeVM repository. VM Operator will then serve as the vSphere provider for KubeVM.

### Website URL

https://github.com/vmware-tanzu/vm-operator/

### Roadmap

https://github.com/vmware-tanzu/vm-operator/blob/feature/kube-vm/external/kubevm/ROADMAP.md

### Roadmap context

The roadmap is ordered by dependency rather than by date, and says so, because several milestones are gated on provider implementations arriving from other organizations.

* **A working generic core** — the reconciler, the provider contract expressed in code, and the companion catalog types the API already references.
* **A second provider** — a KubeVirt provider first, because it is the cheapest possible second implementation: open source, runs in `kind`, needs no cloud account, so portability becomes testable in CI by anyone. EC2 and GCE providers follow, along with a conformance suite defining what "supports KubeVM" means.
* **Orchestration above the machine** — `VirtualMachineTemplate` and `VirtualMachineSet`, `VirtualMachineService`, `VirtualMachineDeployment`, and a bootstrap provider contract.
* **API and project maturity** — capability tiers with provider feature discovery, conversion machinery, and independent release cadence for the core and each provider.

The roadmap also states what is out of scope: re-implementing live migration, high availability, or host placement, all of which remain the provider's responsibility.

### Contributing guide

https://github.com/vmware-tanzu/vm-operator/blob/main/CONTRIBUTING.md

### Code of Conduct (CoC)

https://github.com/vmware-tanzu/vm-operator/blob/main/CODE_OF_CONDUCT.md

### Adopters

_No response_

### Maintainers file

https://github.com/vmware-tanzu/vm-operator/blob/main/MAINTAINERS.md

### Security policy file

https://github.com/vmware-tanzu/vm-operator/blob/main/SECURITY.md

### Standard or specification?

The project is a reusable implementation, not a submission to a standards body. It does, however, include a specification-like element: the provider contract is a defined set of well-known field paths and behaviors that any implementation must satisfy, versioned independently of any one implementation. Expressing that contract in code and defining conformance tests for it are on the roadmap.

### Business product or service to project separation

The sponsoring organization is [Broadcom](https://www.broadcom.com/). KubeVM itself is a vendor-neutral API and set of controllers; it is not offered as a Broadcom product or commercial service, and nothing in the core API is specific to any vendor's platform.

The relationship to a commercial product is confined to one provider. VM Operator, the first provider implementation, is also the control plane for Broadcom's VM Service. That is deliberate as it gives the project a production-proven implementation rather than a prototype. It is just one provider among the EC2, GCE, and KubeVirt providers on the roadmap, and the core API imports no provider code by design.

### Why CNCF?

**A multi-vendor API cannot credibly be owned by one vendor.** KubeVM's value depends entirely on providers from organizations other than the sponsor implementing it. Asking a hyperscaler or a competing virtualization vendor to implement an API governed by Broadcom is a materially harder request than asking them to implement one governed by a neutral foundation.

**The timing is deliberate.** The API is at `v1alpha1` with open design questions still recorded in the repository, so there is real room for a second implementer to change its shape. A design hardened privately and presented finished would offer no such room.

**CNCF provides the venue for exactly the conversations the project needs**. TAG Workloads Foundation, and direct engagement with other hyperscalers / hypervisors / VM platforms about a provider for KubeVM.

The project also expects to use CNCF neutrality to broaden a maintainer base that is currently single-vendor. We wish to share our learnings from managing VMs on Kubernetes clusters using VM Operator and being part of the CNCF community helps bring in more vendors to the table.

### Benefit to the landscape

The landscape currently represents one architectural answer to running VMs under Kubernetes: converging the VM into the container model by running a hypervisor process inside a Pod. That is the right answer where Kubernetes is the only infrastructure layer.

KubeVM adds two things the landscape does not have:

* **A second design point**: a Kubernetes-native front door to a hypervisor-native estate, for the large population of organizations who will not re-platform an existing virtualization investment onto bare-metal, and who want to keep that platform's live migration, storage, and HA rather than re-implement them on raw Kubernetes primitives.
* **A portability layer for VMs**: today a VM manifest is bound to one platform's controller. A portable virtual machine object means higher-level orchestration is written once and works across providers, which is the layer the landscape is missing rather than another VM implementation.

### Cloud native 'fit'

**Kubernetes-native architecture**

* CRDs for the whole API surface: declarative, version-controlled, GitOps-compatible.
* Controller-runtime reconciliation driving observed state toward declared state, with owner references and finalizers for lifecycle and garbage collection.
* Condition-based status, and a condition vocabulary that reports platform capability gaps and retryable failures rather than rejecting writes the API server cannot evaluate.
* Namespace scoping and RBAC as the tenancy boundary; no new authorization mechanism.
* CEL validation in the CRD schema for immutability and grow-only constraints, so invalid transitions are rejected without a webhook.

**Reuse over reinvention**

* Disk provisioning and quality of service reference `StorageClass` and `VolumeAttributesClass` rather than platform-specific IOPS fields, so existing CSI drivers apply unchanged.
* Cross-object references follow Cluster API's versionless contract-reference pattern, so providers can roll their APIs forward without rewriting referring objects.

The only variable relative to a typical CNCF controller project is that the reconciled resource runs outside the cluster's own compute nodes.

### Cloud native 'integration'

**Direct dependencies**

| CNCF Project | How KubeVM uses it |
| :---: | ----- |
| **Kubernetes** | Core dependency. KubeVM is a CRD and controller extension of the Kubernetes API — no fork, no modification, no direct etcd access. |
| **CSI** | Disks reference `StorageClass` for provisioning and `VolumeAttributesClass` for mutable quality of service, so CSI drivers perform the work. The reference provider's data volumes are `PersistentVolumeClaim`-backed. |

**Complementary integrations**

| CNCF Project | Integration type |
| :---: | ----- |
| **KubeVirt** | A target provider, not a competitor, and the first on the roadmap — it needs no cloud account, so it makes the portability claim testable in CI. |
| **containerd / OCI** | OCI-based image distribution is under evaluation as a cross-provider image format, to align VM image distribution with container-native registries. |
| **Cluster API** *(Kubernetes SIG, not CNCF)* | KubeVM adopts its core-plus-providers pattern, contract-versioned references, and duck-typed status contract. The two are complementary layers: Cluster API provisions clusters, KubeVM provisions machines that are not cluster nodes. |

**Ecosystem position**

KubeVM is an abstraction layer, not a virtualization implementation. It does not compete with hypervisors, CNIs, or CSI drivers; it gives the ecosystem one API through which any of them can be driven for VM workloads.

### Cloud native overlap

| Project | CNCF Level | Focus | How KubeVM differs |
| :---: | :---: | ----- | ----- |
| [KubeVirt](https://github.com/kubevirt/kubevirt) | Incubating | Runs a hypervisor process (QEMU/KVM) inside a Pod, so the VM is a cluster workload using the Kubernetes scheduler, CNI, and CSI directly. | KubeVM leaves the VM as a native object of the underlying platform and uses Kubernetes as the API surface, delegating placement, migration, and storage to that platform. The two answer the same question for different starting conditions: Kubernetes as the only infrastructure layer, versus Kubernetes as a front door to an existing one. The relationship is intended to be integration: a KubeVirt provider is on KubeVM's roadmap, so the portable API can run on KubeVirt. |
| [Crossplane](https://github.com/crossplane/crossplane) | Incubating | General-purpose composition and provisioning of external cloud resources through provider packages. | KubeVM is a VM-specific API with a typed lifecycle contract: power state, disks, interfaces, bootstrap, conformance-defined provider behaviour rather than a general resource-composition layer. It also targets on-premises hypervisor estates, which a cloud-API-shaped abstraction serves poorly. |

### Similar projects

Comparison with CNCF projects is covered above. Outside the CNCF:

* **AWS Controllers for Kubernetes (ACK)** and **Azure Service Operator** — deliberately single-vendor facades over one cloud's API. They provide a Kubernetes interface but no portability across platforms, which is the gap KubeVM addresses.
* **virtrigaud**, **kubeswift** — independent attempts at a multi-hypervisor VM API, with limited adoption. KubeVM differs in having a production-proven provider implementation from the outset, and in pursuing a conformance-defined contract rather than a best-effort abstraction.

### Landscape

No.

### Insights

No.

### Trademark and accounts

- [x] If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

### IP policy

- [x] If the project is accepted, I agree the project will follow the CNCF IP Policy

### Will the project require a license exception?

N/A - Project uses Apache 2.0 license

### Project "Domain Technical Review"

Not yet completed. The intent is to present to **TAG Workloads Foundation** and complete the Day 0 portion of the General Technical Review questionnaire.

### Application contact email(s)

arunesh-kumar.pandey@broadcom.com

### Contributing or sponsoring entity signatory information

|Name |Country|Email address |
|---------|-------|---------------------------------|
|Ali Emadi|USA |ali dot emadi at broadcom dot com|

### CNCF contacts

TAG Workloads Foundation TL - Rajas Kakodkar (@rajaskakodkar)

### Additional information

_No response_