[Request]: support KubeSolo for the k8s plugin
- Dominant language
- Swift
- Stars
- 49.9k
- Forks
- 1.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 22
Description
### Feature or enhancement request details
## Summary
Hello! I'm a core maintainer of [KubeSolo](https://github.com/portainer/kubesolo) and would like to contribute support for running it as a `container k8s` node, via `--node-image` pointing at a KubeSolo image.
KubeSolo already runs on Apple Containers through plain `container run`, so this should be a relatively small task. The one blocker is that the bootstrap flow assumes kubeadm, and KubeSolo bootstraps itself.
## Background
KubeSolo is a single-node Kubernetes distribution that runs the API server, controller-manager, kubelet, kube-proxy and containerd in one process.
It has no systemd and does not use kubeadm. It generates its own PKI and bootstraps its control plane directly from its entrypoint.
## Where Apple Containers support stands today
KubeSolo already runs on Apple Containers, and we [document it](https://github.com/portainer/kubesolo/blob/develop/docs/configuration/container-mode.md#running-with-apple-containers):
```bash
container run -d --name kubesolo --cap-add ALL \
--tmpfs /tmp --tmpfs /run \
-v kubesolo-data:/var/lib/kubesolo \
-p 16443:6443 \
portainer/kubesolo:v1.2.0
```
That works, but it is too manual. Once a cluster is bootstrapped, users then have to extract the kubeconfig with `container exec` and hand-edit the server address, and there is no lifecycle management. Instead, we would much rather users simply run:
```bash
container k8s create --node-image portainer/kubesolo:v1.2.0
```
That fits the native workflow you are already building, and would give KubeSolo users on macOS the same one-command experience they get on other engines. Commands like `k8s list`, `start`, `delete` and `write-config` will also be gold!
## What I tried
```bash
container k8s create --node-image kubesolo:k8s-plugin --name kubesolo-dev
```
To satisfy the plugin's bootstrap script I added three stubs to a KubeSolo image:
| Stub | Purpose |
|---|---|
| `/usr/bin/kubeadm` | KubeSolo has already bootstrapped by the time this runs; the stub publishes its existing kubeconfig to `/etc/kubernetes/admin.conf` |
| `/usr/bin/systemctl` | no-op — KubeSolo supervises its own containerd and does not read `/etc/containerd/conf.d` |
| `kubectl` wrapper | drops the kindnet apply — KubeSolo ships its own CNI, and two would conflict |
Result:
```
Waiting for node to boot: ["node": kubesolo-dev]
Preparing node: ["id": kubesolo-dev]
Running kubeadm init: ["node": kubesolo-dev]
Removing control-plane taint for single-node scheduling: ["node": kubesolo-dev]
Removing kubeconfig: ["cluster": kubesolo-dev, "path": /Users/steven/.kube/config]
Error: failed to start process ... in container kubesolo-dev
(cause: "vmexec error: failed to find target executable /bin/kubectl")
```
The node booted, bootstrap completed, the kubeconfig was published, and the flow reached post-bootstrap steps.
## Findings
**1. The containerd readiness check needed no changes.**
KubeSolo already symlinks its socket to `/run/containerd/containerd.sock`, so the plugin's `test -S` passed as-is. Worth noting since it suggests that part of the contract is already generic.
**2. The post-bootstrap steps are not needed either.**
After `kubeadm init` the plugin applies kindnet and removes the control-plane taint, both via `kubectl`.
KubeSolo ships its own CNI, so applying kindnet would leave two CNIs competing for pod networking.
It is also single-node by design and sets no control-plane taint, so there is nothing to remove.
**3. The bootstrap block is the only real coupling.**
Everything around it is distribution-agnostic — boot the node, wait for a runtime, read a kubeconfig back out.
Only the containerd config write, `systemctl restart`, `kubeadm init`, the kindnet apply and the taint removal assume kind specifically.
## Proposal
**Add a way for a node image to declare that it bootstraps itself.**
When set, the plugin would skip the whole bootstrap block:
| Step | Why it can be skipped |
|---|---|
| write `/etc/containerd/conf.d/…` | the distribution configures its own runtime |
| `systemctl restart containerd` | no systemd; the runtime is already supervised |
| `kubeadm init` | the control plane is already up |
| `kubectl apply -f kindnet.yaml` | the distribution provides its own CNI |
| remove control-plane taint | single-node distributions set none |
Instead, wait for a kubeconfig to appear at a declared path, defaulting to `/etc/kubernetes/admin.conf`.
The mechanism is entirely your call: something like a `--bootstrap none` flag, an image label, or probing for a marker file would all work for us. The requirement from our side is pretty much boot the image, wait, read the kubeconfig.
## What I can contribute
- A PR implementing the self-bootstrap hook, following the design you prefer
- Testing a branch, if you'd rather shape it yourselves
- The shim image and steps, if useful for reproduction
If this isn't a direction you want `container k8s` to take, that's completely fair! Knowing either way is valuable, and we'll document the `container run` path for our users instead.
Thanks for considering 😄
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Contributor guide
Research direction
Trace the `container k8s create --node-image` bootstrap flow, focusing on the kubeadm, containerd, systemctl, kindnet, taint-removal, and kubeconfig steps described in the issue. Run the KubeSolo reproduction command or equivalent node-image flow first. Done means a self-bootstrapping image can skip distribution-specific bootstrap work, wait for its declared kubeconfig path, and support the existing k8s lifecycle commands.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, swift
- Domain
- cli, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100