oxidecomputer / oxidecomputer/hubris
Want thermal trip black box data
Open
@cbiffle is already working on this.
Since Oct 17, 2022.
product
service processor
thermal 🚫🔥🚫
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 239
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 23
Description
We recently had a thermal trip event occur because I forgot to install an air-flow shroud. While the system correctly detected that we had a thermal trip and a humility get_state made that very obvious, However, the next question I had was: What was the last temperature we saw?
To answer this I think there are a few things we want to do here in increasing involvement:
- Have some kind of ring buffer of data that we've read so we can denote what the last set of values we saw were. While the last valid measurement is useful, having a few entries that led there will help. This is probably tied into the broader thermal loop restructuring.
- Eventually we want to make sure that we're holding on to data and evacuating it over the management network and transforming it into metrics data (cc @jgallagher, @bnaecker). That may also want to have a different policy of how much data is kept around to cover some amount of network disconnects and other periods were such services are not available. As an example, say we cold start a rack and one system has a thermal trip of the CPU (or really any other component such as a VR), we'd want to keep that around long enough so someone else could contact and discover us and get that information out.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.