oxidecomputer / oxidecomputer/hubris

Want thermal trip black box data

Open
#702 2 comments 0 reactions 1 assignee View on GitHub

@cbiffle is already working on this.

Since Oct 17, 2022.

product service processor thermal 🚫🔥🚫
Dominant language
Rust
Stars
3.6k
Forks
239
Avg merge
1d 12h
Merged PRs (30d)
23

Description

We recently had a thermal trip event occur because I forgot to install an air-flow shroud. While the system correctly detected that we had a thermal trip and a humility get_state made that very obvious, However, the next question I had was: What was the last temperature we saw?

To answer this I think there are a few things we want to do here in increasing involvement:

  • Have some kind of ring buffer of data that we've read so we can denote what the last set of values we saw were. While the last valid measurement is useful, having a few entries that led there will help. This is probably tied into the broader thermal loop restructuring.
  • Eventually we want to make sure that we're holding on to data and evacuating it over the management network and transforming it into metrics data (cc @jgallagher, @bnaecker). That may also want to have a different policy of how much data is kept around to cover some amount of network disconnects and other periods were such services are not available. As an example, say we cold start a rack and one system has a thermal trip of the CPU (or really any other component such as a VR), we'd want to keep that around long enough so someone else could contact and discover us and get that information out.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.