oxidecomputer / oxidecomputer/omicron

Nexus `MgsUpdateDriver` can continue to push an update forward after the planner considers it done

Open
#9,137 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

(This is spawned out of #9133.)

When MgsUpdateDriver starts an update, it inserts it into self.futures and tracks metadata in self.in_progress: https://github.com/oxidecomputer/omicron/blob/4d5bdc6d348b27761348d763c4085f060bcefc18/nexus/mgs-updates/src/driver.rs#L354-L358

If that work pushes the device forward to the point where an inventory collection shows that the update is complete, but the driving future doesn't itself complete yet, it's possible for the planner to remove the update from the set of PendingMgsUpdates (because inventory shows it's done) while the driving future is still running. This has one definite problem: Nexus may start driving a second update in parallel - this happened on #9133. From reading the code there may be another problem: if the future later completes, it looks like we unconditionally put it back into self.delayq and self.waiting: https://github.com/oxidecomputer/omicron/blob/4d5bdc6d348b27761348d763c4085f060bcefc18/nexus/mgs-updates/src/driver.rs#L405-L412

without checking whether it's still present in the config we have.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in nexus/mgs-updates/src/driver.rs at the handling of self.futures and self.in_progress around lines 354-358, then inspect the completion path around lines 405-412. Trace the interaction with inventory, PendingMgsUpdates, self.delayq, and self.waiting, using issue #9133 as context. Done means an update cannot be driven in parallel with work still running and a completed future is not re-added when it is absent from the current configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.