oxidecomputer / oxidecomputer/hubris

SP reset reporting needs work

Open
#1,978 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
3.6k
Forks
239
Avg merge
1d 12h
Merged PRs (30d)
23

Description

In attempting to use the SP's "reset reason" feature in production for the first time, we've noticed that it has... issues.

  1. Because of the RoT's control of the reset line, the reset reason is almost always "Pin" in practice.
  2. This is because the reporting of the reset reason collapses various potential sources into just one, when in fact it should be a set of reasons.
  3. We also clear the reset reason immediately on boot, which means successive resets lose data. The hardware appears to accumulate reset reasons across reboots, so if we get a series of fast reboots we can learn at least some information about all of them. We probably want to make the clearing conditional or delayed.
  4. There is no way to actually ask a production image for the reset reason. It's only exposed in an unused Idol call, so people have been getting it through humility ipc ... which of course is disabled in prod.

My current perspective is that we should do the following:

  • Always report the raw hardware reset bits, alongside a bitset of platform-independent interpreted reasons. This way a curious engineer with a datasheet can map them back to hardware behaviors.
  • Only clear the reset reason when we have reason to believe we're stable. The hackiest way of doing this would be doing it, like, 60 seconds after boot or whatever. A better way would be having the control plane collect and clear the data.
  • Speaking of which, we need a way of getting that data out over the network. Could be hacked into something like gimlet-inspector for now but ought to be available in a more standardized form. Perhaps an ereport?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start by tracing the existing Idol reset-reason call and the humility ipc path, then review how gimlet-inspector or an ereport could expose the data over the network. Done should cover raw and interpreted reasons, safe clearing after reboot, and production access.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
embedded-iot
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.