oxidecomputer / oxidecomputer/propolis

CI: would be nice to test a variety of guests automatically

Open
#1,173 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

testing
Dominant language
Rust
Stars
270
Forks
42
Avg merge
4d 5h
Merged PRs (30d)
6

Description

currently, in CI, PHD is run with the in-tree phd-tests/artifacts.toml, which tells phd about alpine.iso and OVMF_CODE.fd as well as the S3 bucket they get fetched from.

it would be nice to run PHD tests on other non-Alpine guests in CI. personally speaking, in validating changes around device emulation or anything substantially guest-related, I try to at least check out the device on the gamut of Ubuntu, Debian, Windows Server, OmniOS, and FreeBSD (plus Alpine usually) to theoretically cover the most likely guest drivers and kernels that might be driving Propolis.

while there is a Windows adapter for PHD (and Debian + Ubuntu!) we don't run those in CI yet because we don't have a great idea of how to make test images available without promising ourselves a long download time in CI, don't know how much space is available to put test images on (a test Windows is easily a 20g disk image). we ought to figure these things out and actually test other distributions in CI. hopefully such an answer could let us layer on other OSes and cover more of the examples I mentioned above.

practically speaking, test coverage under PHD is mostly "do guest drivers basically work, does the guest kernel basically work"; a collection of Linuxes are only interesting because there are different kernels that LTS images are based on, and different distros carry different patches for the kernel/init/other software which can make for per-distro surprises. testing Ubuntu and Debian and Alpine in CI may not be much more useful run-to-run than just picking one. more to say on this in a different Linux-specific issue though :)

(separate to this is the question of how useful are PHD tests with other guests! https://github.com/oxidecomputer/propolis/issues/1172 and more about that.)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .github/buildomat/phd-run-with-args.sh and phd-tests/artifacts.toml to understand how the current Alpine image and OVMF_CODE.fd are supplied to PHD in CI. Investigate image download time, available CI storage, and the existing Windows, Debian, and Ubuntu adapters. Done means CI can run PHD tests against additional guest operating systems without impractical image or runtime costs.

Written by the indexing model from the issue text.

Assessment

Domain
ci-cd, operating-systems, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.