oxidecomputer / oxidecomputer/dendrite

Want `swadm` subcommand for simulating physically reseating a transceiver

Open
#351 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
20
Forks
3
Avg merge
3d 11h
Merged PRs (30d)
4

Description

In some situations, we'd like to "fully reset" a transceiver. As an example, the Tofino software sometimes fails to fully power up a module. We haven't seen this much for a while, but we did just have another, similar situation at a customer site. In this case, we were also running Omicron's rack-setup process, which means the sled-agent is continually attempting to construct the links we've configured.

This complicates things. swadm sp xcvr reset will fail if the module is in automatic mode, but setting it to manual with swam sp xcvr set-management-mode manual will fail if there are extant links. One could instead use the lower-level xcvradm tool to reset the module, but that's out of band, and the Tofino doesn't reliably restart its module state machines.

We should add a subcommand to swadm and possibly a new Dendrite API, for fully effecting this kind of simulated reseat. It's a little tricky in light of the sled-agent reconciliation mentioned above, but this would essentially do:

  • probably set some bit to prevent API operations on the module
  • store the parameters for all the links on the port
  • delete all the links
  • set the module into management mode
  • reset the module
  • set the module into auto mode
  • recreate all the links
  • unset the bit from the top

I'm not convinced this could be done reliably, but it's worth pursuing. We often have physical access

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing swadm sp xcvr reset and management-mode operations, along with the sled-agent reconciliation behavior described here. Determine whether a swadm subcommand and/or Dendrite API can reliably perform the listed reseat sequence; done means links and module state are restored without conflicting API operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, cli, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.