oxidecomputer / oxidecomputer/omicron

Nexus rejects newer system versions if the git commit is lexographically earlier

Open Beginner friendly
#9,071 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

I tried to do a Nexus-driven update from 8571be38c0ba03a72a9094c32b9825a8cc721cd4 (a commit on main) to 544f608e05ac0f84218338e0c74dc9513295dbde (the next commit on main), and the request to set the target system version was rejected with

Error Response: status: 400 Bad Request; headers: {"content-type": "application/json", "x-request-id": "68a82eaa-0b69-4da0-a9f0-e5eb67d2552a", "content-length": "240", "date": "Wed, 24 Sep 2025 14:55:29 GMT"}; value: Error { error_code: None, message: "The requested target system release (16.0.0-0.ci+git544f608e05a) is older than the current target system release (16.0.0-0.ci+git8571be38c0b). This is not supported.", request_id: "68a82eaa-0b69-4da0-a9f0-e5eb67d2552a" }

The source of this is where we do a comparison between two semver::Versions: https://github.com/oxidecomputer/omicron/blob/2648592fa4c26d4673e3cfca583834c47c008834/nexus/src/external_api/http_entrypoints.rs#L6853-L6862

Because these versions match in their major/minor/patch, we're falling back to comparing the BuildMetadata, which is ordered lexicographically:

However for comparing build metadatas among one another, they do have a total order which is determined by lexicographic ordering of dot-separated components. Identifiers consisting of only digits are compared numerically. Otherwise, identifiers are compared in ASCII sort order. Any numeric identifier is always less than any non-numeric identifier.

We discussed a bunch of options for unblocking testing:

  • Relax this check to ignore the build metadata
  • Add a flag to the external API to override the version check in some way (e.g., "override: ignore build metadata")
  • Add an internal API to set the target release version, and give it an override flag

We're going to go with option 1; it's not any less safe than what we have today (where you can upgrade from commit to commit if the build metadata sometimes) and is very easy to implement.

At the update watercooler, we also discussed ways to make this more closely match requirements we'd actually like to enforce, such as upgrading from a commit on main to a newer commit on main (but not the other way around); upgrading from a release branch back to main (as long as the main commit is newer than the point of main where the release branch was forked); etc. A bunch of edge cases and subtleties came up in this discussion. @davepacheco plans to write an RFD to discuss this, but it's not an R17 ship blocker.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in nexus/src/external_api/http_entrypoints.rs around lines 6853-6862, where the two semver::Version values are compared for the target system release. Reproduce the Nexus-driven update from 8571be38c0ba03a72a9094c32b9825a8cc721cd4 to 544f608e05ac0f84218338e0c74dc9513295dbde, then verify that the request is accepted despite the lexicographically earlier build metadata.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, rust
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.