prometheus / prometheus/client_rust

Performance regresssion with (&'static str, &'static str) label set.

Open
#248 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
606
Forks
113
Avg merge
9h 7m
Merged PRs (30d)
8

Description

Hi, thanks for building the Prometheus client in Rust. Really love it.

When doing the benchmark, I added another test case with (&'static str, &'static str) label set as follows:

    c.bench_function(
        "counter family with (&'static str, &'static str) label set",
        |b| {
            let family = Family::<(&'static str, &'static str), Counter>::default();

            b.iter(|| {
                family.get_or_create(&("method", "GET")).inc();
            })
        },
    );

Considering there should be no dynamic allocation when accessing the metric with (&'static str, &'static str) label set, I suppose the performance would be the same as the custom type label set.

But the benchmark turned out oppositely. (27ns vs 15ns on avg)

     Running benches/family.rs (target/release/deps/family-f5815167dcf70f4a)
Gnuplot not found, using plotters backend
counter family with [(&'static str, &'static str)] label set
                        time:   [41.655 ns 42.005 ns 42.483 ns]
                        change: [-1.0231% +1.7906% +7.1726%] (p = 0.51 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe

counter family with Vec<(&'static str, &'static str)> label set
                        time:   [62.579 ns 63.896 ns 65.827 ns]
                        change: [-1.8609% +0.9648% +4.0991%] (p = 0.58 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe

counter family with (&'static str, &'static str) label set
                        time:   [21.716 ns 21.771 ns 21.833 ns]
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

counter family with Vec<(String, String)> label set
                        time:   [204.69 ns 208.56 ns 214.56 ns]
                        change: [-5.1373% -3.8646% -2.4135%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

counter family with custom type label set
                        time:   [14.648 ns 15.786 ns 17.877 ns]
                        change: [-8.7872% -2.4706% +6.7788%] (p = 0.61 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

I'm not sure where the regression comes from. Is it as expected or did I miss something?

Thanks again for your help. 🙏

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the benchmark in benches/family.rs and trace Family::get_or_create for tuple and custom labels. Re-run the reported release benchmarks, then determine whether the timing difference is expected and document the cause or provide a confirmed fix with updated benchmark results.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.