randomparity / randomparity/kdive
Evaluate mkosi as a future minimal, uniform rootfs build lane (parked research)
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 1h 26m
- Merged PRs (30d)
- 311
Description
## Summary
Evaluate **mkosi** (systemd project) as a future *minimal, uniform, from-scratch* rootfs
build lane for the local-libvirt provider — an optional alternative to the current
vendor-cloud-image + `virt-customize` (libguestfs) build plane, for the subset of cases that
don't need vendor-cloud-image fidelity.
This issue exists to **preserve the research** from the "cloud-init vs. roll-our-own" design
discussion. It is **not** the near-term direction — the near-term decision is to *embrace
cloud-init natively on the vendor cloud images* (separate spec/ADR). mkosi is parked here as a
deliberate future option.
## Background / why this came up
The debian-family rootfs shipped SSH-unreachable (live-proven 2026-07-01): the build disables
cloud-init but installs no replacement NIC network config, so the guest boots with no IP and
sshd never answers (`Connection timed out during banner exchange`). That exposed a broader
question: every distro base we use ships cloud-init, yet we mask it and hand-roll per-family
first-boot glue (NetworkManager keyfile on rhel, `kdive-sshd-keygen` + `--ssh-inject` on both,
cloud-init masking, machine-id seed). We reconsidered the whole premise before committing.
## Options evaluated
### Chosen (near-term, separate spec): embrace cloud-init on the vendor cloud images
Keep the vendor Fedora Cloud / Rocky-CentOS GenericCloud / Debian genericcloud images (highest
fidelity, esp. for the RHEL family where the userland is often the actual target). Provide
cloud-init a NoCloud datasource so it runs deterministically and owns network + host keys +
authorized_keys uniformly across all families. Delete the hand-rolled first-boot glue. Keeps
the existing `virt-customize` build plane; no new heavyweight tool.
### Rejected: osbuild / image-builder (osbuild.org)
The native Fedora/RHEL image builder, but **disqualified for kdive**:
- **No Rocky/Alma targets** — it builds `fedora-N` / `centos-N` / `rhel-N` only; kdive ships
Rocky 8/9/10 (#823), which osbuild cannot produce.
- **Host/entitlement constraints** — "you cannot build an OS image that differs from the
image-builder host." From a Fedora host: Fedora + CentOS Stream OK; **RHEL proper needs an
RHSM subscription** (or an experimental UBI-container bootstrap that was *broken on Fedora 43
for RHEL 10.1* due to post-quantum RPM signing keys, osbuild/image-builder-cli#413).
Same-arch only; needs root + loopback + mount.
- **No fidelity gain** — it *composes* a fresh image from RPM repos, so it does not start from
the vendor's published cloud qcow2; our current virt-customize-on-GenericCloud is *more*
vendor-faithful. Net: it would cover only part of one family and fragment the build, opposite
of the unification goal.
### Parked (this issue): mkosi
A systemd-project tool purpose-built for "assemble a test VM, boot it under qemu, ssh in" —
exactly the shape of kdive's local-libvirt lane (`mkosi vm` + `mkosi ssh`, `Ssh=`/`mkosi
genkey` bake the key declaratively).
**Attractive for a future minimal-lane subset:**
- Uniform across **all** kdive distros in one tool: fedora, rocky, alma, centos, debian, ubuntu
(also arch/opensuse). Covers the Rocky gap osbuild can't.
- **systemd-native first-boot** (systemd-networkd + declarative ssh keys) → **cloud-init
disappears entirely**; nothing to fight or seed.
- Fast, incremental, unprivileged builds; INI/systemd-style declarative config with profiles
and drop-ins; used by the systemd project for cross-distro CI.
- Output formats incl. raw GPT disk / qcow2 / directory / cpio / UKI — fits direct-kernel boot.
**Why it is NOT the near-term choice:**
- Builds **minimal images from packages**, not the vendor cloud image → **loses the RHEL/Fedora
vendor-userland fidelity** we want when triaging whether an issue is even a kernel issue.
- Bigger migration: replaces the whole `rootfs_build` plane and the `FamilyCustomizer` argv
model, not just first-boot.
- Operational: building Debian/Ubuntu from a Fedora host requires `apt` + `dpkg` +
`debian-archive-keyring` installed on the host (recent mkosi runs apt/dpkg on the host;
debootstrap alone is insufficient — systemd/mkosi#1306, #2220); bootable images need
`ukify`/systemd tooling.
**Where mkosi would be worth it later:** a fast, minimal, uniform "just needs to work" lane for
non-vendor-specific testing (e.g. generic kernel-under-test hosting, cross-distro smoke matrices,
CI images) where a stock cloud userland is not required.
## Reference links
- mkosi: https://github.com/systemd/mkosi ·
https://0pointer.net/blog/a-re-introduction-to-mkosi-a-tool-for-generating-os-images.html
- mkosi distro/host requirements: systemd/mkosi#1306, systemd/mkosi#2220
- osbuild / image-builder: https://osbuild.org/docs/ ·
cross-distro breakage: https://github.com/osbuild/image-builder-cli/issues/413
- Debian SystemBuildTools survey: https://wiki.debian.org/SystemBuildTools
## Acceptance / next step
No code now. Revisit when a minimal, distro-uniform, non-vendor-fidelity image lane is needed;
at that point spike `mkosi vm`/`mkosi ssh` for one rhel + one debian target on the Fedora build
host and compare boot-to-ssh time and image size against the cloud-image lane.
Contributor guide
Research direction
When this is revisited, start by reviewing the mkosi documentation and the existing cloud-image lane. Spike mkosi vm and mkosi ssh for one RHEL-family target and one Debian target on the Fedora build host, then compare boot-to-SSH time and image size against the cloud-image lane.
Written by the indexing model from the issue text.
Assessment
- Domain
- build-system, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100