rust-lang / rust-lang/rustc-perf
perf triage classification of PR 89611 was "probably changed"; why not definite?
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 733
- Forks
- 189
- Avg merge
- 10h 52m
- Merged PRs (30d)
- 43
Description
Looking at the performance report linked here:
| Benchmark & Profile | Scenario | % Change | Significance Factor ? |
|---|---|---|---|
| encoding opt | full | -0.80%? | 2.21x |
| encoding opt | incr-full | -0.72%? | 2.36x |
| coercions check | incr-patched: println | -0.64% | 4.82x |
| helloworld debug | incr-full | -0.53%? | 2.07x |
| helloworld debug | incr-unchanged | -0.52% | 4.79x |
| coercions check | incr-unchanged | -0.51% | 7.92x |
| helloworld debug | full | -0.45%? | 1.83x |
| coercions opt | incr-unchanged | -0.43% | 5.67x |
| ripgrep opt | incr-unchanged | -0.42% | 2.73x |
| cranelift-codegen opt | incr-unchanged | 0.38%? | 1.81x |
| coercions debug | incr-unchanged | -0.36% | 3.87x |
| issue-58319 debug | incr-unchanged | -0.34% | 2.09x |
| token-stream-stress debug | full | -0.31% | 1.74x |
| token-stream-stress debug | incr-unchanged | -0.31% | 2.25x |
| coercions doc | full | -0.25% | 2.89x |
| unicode_normalization check | incr-unchanged | -0.24% | 3.79x |
| coercions check | full | -0.24% | 3.79x |
| issue-58319 debug | incr-full | -0.24% | 2.00x |
| tuple-stress debug | incr-unchanged | -0.24% | 1.94x |
| helloworld opt | incr-patched: println | -0.21% | 2.55x |
| helloworld opt | incr-unchanged | -0.20% | 2.19x |
| coercions check | incr-full | -0.20% | 5.63x |
why isn't this classified as a definite improvement in the performance triage script?
Yes, a lot of them have question marks on them (though I'm not sure the performance triage script includes the question marks in its weighting at this point, based on what I saw elsewhere), but even if you remove the question mark cases, you're still left with e.g.
| Benchmark & Profile | Scenario | % Change | Significance Factor ? |
|---|---|---|---|
| coercions check | incr-patched: println | -0.64% | 4.82x |
| helloworld debug | incr-unchanged | -0.52% | 4.79x |
| coercions check | incr-unchanged | -0.51% | 7.92x |
which seems like it should have been enough for being classified as a performance win, not a "probably" ?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the performance triage script and the linked performance report for PR 89611. Trace how question marks, percentage changes, and significance factors are weighted, then compare that logic with the listed scenarios; done means explaining why this report received a “probably changed” classification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100