oxidecomputer / oxidecomputer/omicron

RSS hands out "reserved" addresses to control plane services

Open
#7,467 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

I'm not sure there's anything particularly urgent here, but it came up on a PR recently so I wanted to note it.

When Reconfigurator is handing out underlay addresses with a sled's subnet to new control plane services it's adding to the blueprint, it skips the first 32 addresses based on this constant: https://github.com/oxidecomputer/omicron/blob/1a0bb7424a04fa221178ef958a49dd492c96b50b/common/src/address.rs#L169-L176

Based on the first half of that comment, I believe the intent here is that we want to reserve some underlay IPs for when the sled itself needs to make autonomous decisions about an IP to use without information from Nexus; e.g., it assigns ...::1 to the global zone, and ...::2 to the switch zone, both of which must exist prior to Nexus being online (or even RSS running).

However, the second half of that comment is basically the source of this issue: when RSS is choosing IPs for control plane services, it only reserves the first two (for the gz and for the switch zone): https://github.com/oxidecomputer/omicron/blob/1a0bb7424a04fa221178ef958a49dd492c96b50b/sled-agent/src/rack_setup/plan/service.rs#L793-L807

I think this means:

  • The constant Reconfigurator is using (SLED_RESERVED_ADDRESSES) makes it look like we have up to 32 addresses for sled-autonomous IPs
  • In practice, we only have two, and the IPs Reconfigurator thinks of as reserved may actually be in use in ways that it could have never assigned

I don't think this has any effect other than confusion as long as we only need the two sled-autonomous IPs we have today. If we ever need to add a third, we'll have to address this somehow (use Reconfigurator to shift all the control plane services up into the non-reserved range, or pick a different reserved range for the new IPs, or ...).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read common/src/address.rs around SLED_RESERVED_ADDRESSES and sled-agent/src/rack_setup/plan/service.rs around the RSS allocation logic, then compare their reserved-address assumptions. Confirm which addresses are intended for sled-autonomous use and define a consistent allocation rule; done means Reconfigurator and RSS agree without consuming one another’s reserved range.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, networking
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.