rust-lang / rust-lang/rustc-perf
Noise measures aren't helpful for noisy metrics
Open
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 733
- Forks
- 189
- Avg merge
- 10h 52m
- Merged PRs (30d)
- 43
Description
Metrics that are inherently noisy are always classified as being noisy (i.e., every benchmark for wall-time is marked as noisy). This is true, but not very helpful. Perhaps we need some per-stat based idea of what noise is.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by locating the noise classification for benchmark metrics and trace how wall-time statistics are handled. Define what per-stat noise means and how the classification should become more useful, then add coverage for the resulting behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100