oxidecomputer / oxidecomputer/omicron
End-to-end CI test should yield instance serial console output on failure
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
The failure stack shows that the test expired here: https://github.com/oxidecomputer/omicron/blob/6d73cf8d856e82883886dade41c956d2fd83034e/end-to-end-tests/src/instance_launch.rs#L113-L142
In other words, the test wanted to see the END SSH HOST KEY KEYS marker on the serial console within 5 minutes and failed when it didn't show up.
There are no Propolis logs here (see #4490), but the sled agent logs show that Propolis at least told sled agent that the guest was running:
SledAgent (InstanceManager): updated state after observing Propolis state change
file = sled-agent/src/instance.rs:354
instance_id = f8252e1e-d83a-44b4-b0ef-d218702bc8c9
new_instance_state = InstanceRuntimeState { propolis_id: Some(a965b005-c60d-43b4-8a01-78fe39d882ed), dst_propolis_id: None, migration_id: None, gen: Generation(3), time_updated: 2023-11-10T22:53:25.027228194Z }
new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(3), time_updated: 2023-11-10T22:53:32.750978136Z }
propolis_id = a965b005-c60d-43b4-8a01-78fe39d882ed
In situations like this, we'd like to be able to see what the test did see on the serial console so that we can assess where the guest got stuck. (From the virt stack's perspective, control has been handed over to the guest here, and we need to know what it was doing: did it manage to boot into the guest OS at all? If so, what did the guest do after that?)
If this test fails after trying to start the guest, the failure path should dump the guest serial console contents (if any) to an artifact that we can retrieve from the logs or output artifacts.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read end-to-end-tests/src/instance_launch.rs around lines 113-142 and inspect the failure path after starting the guest. Run or reproduce the end-to-end CI test, then verify that a failure preserves any guest serial-console contents in a retrievable log or output artifact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ci-cd, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100