Consider passing -Awarnings (or similar) to avoid false alarms from lint *reporting*

See https://github.com/rust-lang/rust/pull/117772#issuecomment-1951430297, quoted here to sidestep github's collapsing of comments:

On a closer inspection, the results for ripgrep, regex and html5ever are quite misleading. Those crates actually have redundant imports, so the lint reporting machinery (lint level checking, diagnostic printing) is invoked now, and that's what is causing regressions, not the check_unused pass itself.

The unused-warnings regression is legitimate, but it's a stress test for imports specifically.

I think we can merge this after addressing #117772 (comment).

There were a number of benchmarks that starting hitting a lint and thus starting exercising lint reporting machinery that was previously uninvoked.

Now: If a lint check is expensive to exercise, then we should measure that as part of benchmarking.

But if a new lint fires and causes new diagnostic code paths to run, that sounds like noise to me.

It would be better, IMO, if we silenced the diagnostic machinery from running at all in such cases, except for isolated benchmarks that are themselves dedicated to exercising the efficiency of the diagnostic machinery.

What do others think?

Start by reading the linked rust-lang/rust pull request and discussion, then inspect how rustc-perf invokes benchmarks and handles lint-triggering diagnostics. Done means ordinary benchmarks no longer exercise newly firing diagnostic reporting paths, while benchmarks dedicated to diagnostic efficiency still measure them.