oxidecomputer / oxidecomputer/omicron
Tracking issue for update related health checks
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
Now that we have the contact_support field in place within the update/status endpoint with a minimal set of health checks, we should have a list of health checks we want to implement, but are less urgent.
Using the list from https://github.com/oxidecomputer/omicron/issues/9876, these are the health checks we'd like to implement
| Health check | Scope | Priority | Implemented |
|---|---|---|---|
| missing zpools | per sled | high | no |
| stale last blueprint (20 mins) | per rack | high | no |
| planner zone checks | per rack | high | |
| SMF service health | per sled/zone | high | yes https://github.com/oxidecomputer/omicron/pull/10271 |
| NVME firmware version | per sled | low | |
| expected physical disk count | per sled | medium | |
| zpool health | per sled | high | yes https://github.com/oxidecomputer/omicron/pull/10271 |
| unmounted dataset | per sled | medium - Will need to make sure the datasets are ones that matter (e.g. not support bundles) |
|
| expected crucible zone count | per rack | high | |
| dataset usage over 80% of quota or root dataset with <150 GiB avail |
per sled | medium | |
| crucible free space when total capacity minus total used < 5TiB |
per rack | medium | |
| dimm expected memory (1 or 2 TB) | per rack | low | |
| missing sleds | per rack | high | yes https://github.com/oxidecomputer/omicron/pull/10476 |
| missing components for MGS driven updates |
per rack | medium | |
| stale sagas | per rack | high | yes https://github.com/oxidecomputer/omicron/pull/10271 |
| Kernel low memory scan | per sled | low |
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the update/status endpoint and its existing contact_support health checks, then review issue #9876 and the linked planner-zone issue #10434. Break the unchecked rows into concrete follow-up work, confirm each check's per-sled or per-rack scope and priority, and update the tracking list as implementations land.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100