oxidecomputer / oxidecomputer/omicron

BootstrapAgent::sled_reset doesn't fully reset the sled

Open
#3,820 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

The bootstrap agent exposes an API to reset the sled, and it notes its intent: https://github.com/oxidecomputer/omicron/blob/80b5f7f2f9253695f13832c62dface4116dbb2e8/sled-agent/src/bootstrap/agent.rs#L983-L984

However, it doesn't currently fully reset the state:

  1. There's no way to "uninitialize" the LRTQ salt which gets set in request_sled_agent. (See #3818 - once request_sled_agent has been called once with a rack_id, all subsequent calls must provide the same rack_id).
  2. (I think?) It does not unadvertise the sled prefix that is advertised in request_sled_agent.
  3. (Unsure) Does it clean up any configuration it has sent to the switches (e.g., NAT entries for NTP/external DNS/instance zones)?

I chatted with @andrewjstone briefly and he suggested we may want to do something more extreme: remove anything written to the M.2s (bootstore / ledger), still remove persistent storage as reset_sled does today, then reboot the sled. That would take care of in-memory caches like LRTQ (unsure if there are more of these I've missed); I'm not sure about undoing switch configurations.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in sled-agent/src/bootstrap/agent.rs at the reset_sled implementation and request_sled_agent, then review issue #3818 for the LRTQ salt behavior. Trace the sled prefix advertisement and switch configuration paths, including persistent bootstore/ledger state. Done means the chosen reset design clears all relevant state and its behavior is covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend, infrastructure
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.