Strip `nomodeset nofb gfxpayload=text` from GRUB cmdline so virtio/KMS GPUs probe (BM-emulator + fake-GPU)

Open
#159 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
55/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
linux, shell, ubuntu

Research direction

Inspect the relevant diskimage-builder element's finalise.d scripts and search for where GRUB_CMDLINE_LINUX_DEFAULT is set. Check /etc/default/grub and the update-grub step, then build an Ubuntu 22.04 image and verify that the three flags are absent so KMS and virtio_gpu can probe successfully.

Written by the indexing model from the issue text.

Description

Summary

The Ubuntu 22.04 kube images produced by this repo (verified up through release 2026.04-6, ubuntu-22.04-v1.34.7.qcow2) ship with GRUB_CMDLINE_LINUX_DEFAULT containing:

nomodeset nofb gfxpayload=text

These flags tell the Linux kernel to refuse loading any KMS GPU driver. That is fine for headless serial-console-only servers, but it breaks any workload that expects a working /dev/dri/cardN, including:

  • BM hosts (or BM-emulator VMs) with a virtio-gpu / virtio-vga device attached.
  • Any device-plugin that advertises GPU character devices (e.g. squat/generic-device-plugin exposing /dev/dri/card* as vexxhost.com/fake-gpu, or NVIDIA's plugin in environments where the host exposes a virtual GPU via virtio).

Reproduction (downstream consumer)

In Atmosphere's ironic_emulator_nodes BM-on-Ironic test rig (vexxhost/atmosphere, branch bm-mcapi-fixes):

  1. Boot a Magnum/CAPI cluster onto BM-emulator nodes whose libvirt domains include a virtio-vga device, using image ubuntu-2204-kube-v1.34.7-raw from this repo's release 2026.04-6.
  2. Apply a fake-GPU device-plugin DaemonSet that watches /dev/dri/card*.
  3. Observed on every worker:
    • dmesg | grep -i virtio_gpu shows virtio_gpu: probe failed with error -22.
    • /dev/dri is empty — no card0 device file.
    • kubectl get nodes -o jsonpath='{...allocatable.vexxhost\.com/fake-gpu}' returns 0 on every node.
    • Pods requesting vexxhost.com/fake-gpu stay Pending indefinitely.

Workaround in use today

We do this on each worker after the cluster is up:

nsenter -t 1 -m -u -i -n -p \
  sed -i 's/ nomodeset//g; s/ nofb//g; s/ gfxpayload=text//g' /etc/default/grub
nsenter -t 1 -m -u -i -n -p update-grub
nsenter -t 1 -m -u -i -n -p systemctl --no-block reboot

After reboot, virtio_gpu probes successfully, /dev/dri/card0 appears, and the device-plugin advertises vexxhost.com/fake-gpu=1. Demo pod sees 06:00.0 VGA compatible controller: Red Hat, Inc. Virtio 1.0 GPU (rev 01) and /dev/dri/card0.

This is reliable but has obvious downsides: privileged in-cluster mutation, requires a reboot, and has to be re-applied on every new node (autoscaler, scale-up, rolling upgrade replacing a worker).

Proposed fix

At image-build time, strip the three flags before update-grub:

sed -i 's/ nomodeset//g; s/ nofb//g; s/ gfxpayload=text//g' /etc/default/grub
update-grub

Most natural place is in the relevant DIB element's finalise.d (or wherever GRUB_CMDLINE_LINUX_DEFAULT is currently being set with these flags). If the flags are coming from the upstream Ubuntu cloud image base (i.e. not added by an element here) we can just unconditionally normalise them in a cluster-api-vexxhost-style finalise script.

If keeping the flags is desirable for some "headless serial-only" target, gate behind a build env var (e.g. DIB_DISABLE_KMS=1), default to enabling KMS.

Why "BM GPU" rather than just "fake-GPU"

The same issue will hit any consumer that expects a working DRM device on the kube nodes — virtio-vga in BM-emulator rigs, real GPUs passed through via VFIO on real BM hosts, virgl/QXL in nested-virt CI, etc. The fix is the same regardless of which GPU is attached: don't tell the kernel to refuse KMS in the first place.

References

Happy to send a PR if there's agreement on the approach.

Dominant language
Shell
Stars
12
Forks
9
Avg merge
1m
Merged PRs (30d)
4

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from vexxhost/capo-image-elements

All issues in vexxhost/capo-image-elements

Similar issues

More Shell/Bash issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.