oxidecomputer / oxidecomputer/hubris
Manage fan watchdog state: suppress in A2, enable in A1/0
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 239
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 23
Description
This is a proposal for a partial fix to the "unexpected fan spinup is really loud and annoying" issue.
One of the things that causes unexpected fan spinups is flashing the SP over SWD. This is particularly problematic when it's a remote user in control, since people physically near the machine being reflashed don't know to expect "lol cooling machine go brrrr."
In a discussion this morning, I observed that we probably shouldn't be reflashing the SP in A0. This is convenient since we only really need the fans to go full-bore, and thus need the fan watchdog, in A0. However, currently, the watchdog is unconditionally enabled in all power states, which seems overly conservative.
So, what if we did this?
- On SP boot (which currently implies an A2 transition since we don't support hot SP reset yet), reach out over I2C and ensure the watchdog is off.
- In the sequencer task, before turning up the A1/A0 power rails, reach out over I2C and turn the watchdog on.
- Again in the sequencer task, when transitioning out of A0, after the rails are off, reach out over I2C and turn the watchdog back off.
In this world, a remote user could avoid the wrath of their peers by making sure the machine is "off" (but not really off off) before flashing over SWD.
Note that flashing over the network doesn't cause any of these issues, so for people who have management network connections, you can avoid annoying your coworkers today!
Now, this doesn't do anything to address the design behavior of the watchdog in the event that an I2C bus locks up while we're in A0, so it doesn't completely eliminate the spinup issue, just to be clear.
@nathanaelhuffman @rmustacc
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at SP boot and the sequencer task, then trace the existing I2C fan-watchdog operations across A2, A1, and A0 transitions. Done means the watchdog is off after SP boot, on before the A1/A0 rails rise, and off again after leaving A0 and the rails are down.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- embedded-iot
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100