oxidecomputer / oxidecomputer/omicron
Planner defers all zone updates when any zone is unhealthy
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
We have observed two customer-driven updates where the zone upgrade portion does not begin due to a failed U.2 drive.
The planner correctly determines that the zpool is unavailable and its associated crucible is down:
a93cdff6-bf17-420f-ae9d-9b91a0f3fb2f crucible c262cbe0-60f6-4881-a39a-11b5522d3e26 zone's transient root dataset is not available: oxp_f27830df-8e50-4c3c-888b-056488ff2bb8/crypt/zone/oxz_crucible_c262cbe0-60f6-4881-a39a-11b5522d3e26
The SP and host OS updates are performed, but all of the zone upgrades are deferred until the problem disk and its associated zone are expunged:
|18.0.0-0.ci+gitd211c40e1e5 |19.2.0-0.ci+gite4b75dde134 |error
------------------+---------------------------+---------------------------+------
RoT bootloader |0 |28 |0
RoT |0 |28 |0
SP |0 |28 |0
Host OS (phase 1) |0 |24 |0
Host OS (phase 2) |0 |24 |0
Zone |282 |0 |1
It would be preferable for the system to proceed with the zone updates that are not impacted by the hardware failure.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in nexus/reconfigurator/planning/src/planner.rs at lines 1644-1660, where zone upgrades are deferred when a zone is unhealthy. Trace how the failed U.2 drive and unavailable crucible affect the planner's zone decisions. Done means unaffected zones proceed with their upgrades while the impacted zone remains deferred.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100