oxidecomputer / oxidecomputer/omicron
Oximeter: some crucible_upstairs queries broken on r2
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
Here's an innocuous oxql query that doesn't work on r2, as of this writing:
$ joshcarp@j0sh ~/c/oxidereceiver (jmcarp/remote-write) [1]> oxide --profile r2 experimental system timeseries query --query "get crucible_upstairs:write | filter timestamp > @now() - 15m"
error
Error Response: status: 400 Bad Request; headers: {"content-type": "application/json", "x-request-id": "1dab0d96-4696-4462-973b-3ef4a4852f7c", "content-length": "466", "date": "Mon, 30 Mar 2026 19:21:01 GMT"}; value: Error { error_code: Some("InvalidRequest"), message: "Cannot compute a delta, the timestamp of the next sample has a new start time, or overlaps with the last processed sample. \n epoch start time = 2026-03-17 20:18:41.611418768 UTC\n last timestamp = [2026-03-30 19:20:45.226962784 UTC, 2026-03-30 19:20:55.227868499 UTC]\n new timestamp = [2026-03-17 21:23:37.295694225 UTC, 2026-03-30 19:06:05.191748974 UTC]", request_id: "1dab0d96-4696-4462-973b-3ef4a4852f7c" }
The identical query works on r3, and interestingly also works on r2 if we append | last 1 to the end.
I don't think the relevant code has changed recently, so I'm guessing we're dealing with some unexpected data that's specific to r2, but haven't had time to look yet.
cc @bnaecker
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the experimental system timeseries query command on r2 and r3 with the crucible_upstairs:write query, then compare it with the r2 query ending in | last 1. Start from the reported InvalidRequest timestamp ranges and identify the r2 data or processing difference; done means the original query works on r2 without the workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100