oxidecomputer / oxidecomputer/omicron
`KstatSampler` could report missing self-stat samples
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 572
- Forks
- 97
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 96
Description
The KstatSampler object is used to publish oximeter statistics based on illumos kstats. It includes a few specific bits of information about its own inner workings, such as the number of samples dropped for each tracked object. It places those on a queue, from which oximeter collects data periodically. That's a tokio::sync::broadcast::Sender, which acts like a limited ringbuffer to avoid egregious memory consumption if oximeter doesn't come knocking frequently enough.
In that case, where we've missed samples, the sampler currently logs a message:
It would be nice to record this as its own timeseries, something like kstat_sampler:self_stat_samples_dropped. This needs to be a generic number of self-stat samples we've dropped, not attached to any particular tracked target. At the point we get this error, we don't have the actual dropped Sample, only the count, so it's a general-purpose thing about the KstatSampler itself, not any target it tracks.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in oximeter/instruments/src/kstat/sampler.rs around lines 1319-1326, where dropped self-stat samples are currently logged. Trace how KstatSampler publishes its own statistics, then add a generic kstat_sampler:self_stat_samples_dropped timeseries using the dropped count rather than a tracked target. Done means the sampler records these missed self-stat samples as their own timeseries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100