oxidecomputer / oxidecomputer/hubris
Duplicate hardware metrics on sensor ambiguity
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 239
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 23
Description
Consider the follow oxql query:
> oxide --profile r3 experimental system timeseries query --query 'get hardware_component:temperature | filter component_id == "PSU0" && timestamp > @now() - 1m | last 3'
{
"tables": [
{
"name": "hardware_component:temperature",
"timeseries": [
{
"fields": {
"chassis_model": {
"type": "string",
"value": "913-0000003"
},
"component_kind": {
"type": "string",
"value": "mwocp68"
},
"chassis_serial": {
"type": "string",
"value": "BRM45220018"
},
"chassis_revision": {
"type": "u32",
"value": 8
},
"description": {
"type": "string",
"value": "PSU 0 MCU"
},
"hubris_archive_id": {
"type": "string",
"value": "a7cec194aa73a90f"
},
"chassis_kind": {
"type": "string",
"value": "power"
},
"rack_id": {
"type": "uuid",
"value": "48759651-d754-43e2-ad49-64dde3adc1c2"
},
"sensor": {
"type": "string",
"value": "psu0mcu"
},
"gateway_id": {
"type": "uuid",
"value": "dfeef10e-0dda-432d-a06a-f103456881ce"
},
"component_id": {
"type": "string",
"value": "PSU0"
},
"slot": {
"type": "u32",
"value": 0
}
},
"points": {
"timestamps": [
"2026-08-11T16:23:50.684568706Z",
"2026-08-11T16:23:50.684583584Z",
"2026-08-11T16:23:50.684595417Z"
],
"values": [
{
"metric_type": "gauge",
"values": {
"type": "double",
"values": [
15.75,
45.25,
40.75
]
}
}
]
}
},
{
"fields": {
"hubris_archive_id": {
"type": "string",
"value": "a7cec194aa73a90f"
},
"chassis_revision": {
"type": "u32",
"value": 8
},
"chassis_serial": {
"type": "string",
"value": "BRM45220018"
},
"sensor": {
"type": "string",
"value": "psu0mcu"
},
"component_kind": {
"type": "string",
"value": "mwocp68"
},
"component_id": {
"type": "string",
"value": "PSU0"
},
"rack_id": {
"type": "uuid",
"value": "48759651-d754-43e2-ad49-64dde3adc1c2"
},
"chassis_model": {
"type": "string",
"value": "913-0000003"
},
"description": {
"type": "string",
"value": "PSU 0 MCU"
},
"chassis_kind": {
"type": "string",
"value": "power"
},
"gateway_id": {
"type": "uuid",
"value": "7cff00d5-b96b-4f9f-b6dd-cb8f8d5e42ba"
},
"slot": {
"type": "u32",
"value": 0
}
},
"points": {
"timestamps": [
"2026-08-11T16:23:49.683031412Z",
"2026-08-11T16:23:49.683044597Z",
"2026-08-11T16:23:49.683053594Z"
],
"values": [
{
"metric_type": "gauge",
"values": {
"type": "double",
"values": [
15.875,
45.25,
40.75
]
}
}
]
}
}
]
}
]
}
In particular, the timestamps of the last three values:
"timestamps": [
"2026-08-11T16:23:50.684568706Z",
"2026-08-11T16:23:50.684583584Z",
"2026-08-11T16:23:50.684595417Z"
],
These timestamps are the same down to the millisecond. But we sample the sensor every 1s. Why are we getting multiple almost-identical timestamps for a single sensor, with very different values?
This seems to be happening because the psu0mcu sensor on the mwocp68 actually includes multiple sensors, and we don't distinguish among them in hubris. If the different sensors measured the same thing, this might not matter much, but I believe they're actually measuring different things (ambient vs hot-spot temperatures). So I think these metrics are basically uninterpretable until we include the full sensor names.
It looks like we would fix this by updating metadata around https://github.com/oxidecomputer/hubris/blob/9e553c76105497896a725b89558b46fda0ee10c6/app/psc/base.toml. It would also be helpful to include some kind of check to avoid duplicate sensor names if possible, because we also have a similar conflict in hardware_component:fan_speed, also for the mwocp68.
More selfishly, I noticed this because I tried to ingest these metrics into prometheus, which only has millisecond timestamp precision, and hit errors due to conflicting timestamps. That isn't necessarily an Oxide bug, but I think the missing sensor information actually is a bug.
Possibly relevant to your interests @hawkw @evan-oxide.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting app/psc/base.toml and the metadata for the mwocp68 psu0mcu sensor, then reproduce the hardware_component:temperature query from the issue. Trace how sensor names become metric series and check the similar hardware_component:fan_speed conflict; done means the distinct sensors have full, non-duplicated names and the reported metrics are distinguishable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- embedded-iot, observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100