rust-lang / rust-lang/rustc-perf
Create initial version of a "dashboard" that shows long-term performance trends
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 732
- Forks
- 189
- Avg merge
- 10h 52m
- Merged PRs (30d)
- 43
Description
This should implement https://github.com/rust-lang/rust/issues/48750, but for starters just containing the FROM-SCRATCH and SMALL-CHANGE usage scenarios.
I imagine this to be a separate page on perf.rlo, showing just two graphs, one for each usage scenario.
Each graph should contain a data point for each stable release (starting a few releases back, maybe with 1.17), the current beta, and the current nightly. The score should be calculated as the geometric mean of all build times for the given usage scenario, as listed in https://github.com/rust-lang/rust/issues/48750.
The main tasks to accomplish this are:
- Add the missing benchmarks
- ripgrep
- webrender
- cargo
- winapi
- stm32f103xx
- encoding-rs (
cargo test --lib --no-run) - clap-rs (
cargo test --no-run) - regex (
cargo test --lib --no-run) - syn (
cargo test --no-run) - futures (
cargo test --test=all --no-run)
- Allow for running benchmarks in a Rust version specific way (so we don't crash when trying to specify
-Cincrementalon versions that don't support it yet) - Setup the new page (e.g. under perf.rlo/dashboard.html)
That page should make sure that all benchmark results for computing a given aggregate result are available, otherwise the result would be misleadingly low.
The new benchmarks should also be displayed on the regular perf.rlo page. Some open questions:
- Do we need to start looking into running tests in parallel on multiple machines (because they are becoming more and more)?
- What to do with the
cargo testbenchmarks? Probably just treat them as separate benchmarks, that is, they would add another row instead of another column:encoding-rsandencoding-rs (test), for example, would not be tied together in any way (although it would be nice if they could keep sharing the same source directory)
@Mark-Simulacrum, @nikomatsakis, thoughts?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the benchmark checklist in the issue and the existing perf.rlo page; review how the FROM-SCRATCH and SMALL-CHANGE results are currently produced. Implement the missing benchmarks and Rust-version-specific execution, then add perf.rlo/dashboard.html with release, beta, and nightly graphs whose geometric means include only complete inputs. Verify that the new benchmarks also appear on the regular perf.rlo page.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-visualization, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100