rust-lang / rust-lang/rust

removing needless `.collect()` in the middle of iterator chain reduces performance significantly

Open
#140,873 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

C-optimization E-needs-mcve I-slow T-compiler
Dominant language
Rust
Stars
119k
Forks
16.1k
PR merge metrics
PR metrics pending

Description

reproduction (not minimal):
https://github.com/cyrgani/regulus/commit/6c6dd758d3eadf0383b22d1acf2d7d9ccf21ac04
command: cargo run --release -- tests/programs/sorting_tests.re

The relevant part is this:
https://github.com/cyrgani/regulus/blob/6c6dd758d3eadf0383b22d1acf2d7d9ccf21ac04/regulus/src/state.rs#L60-L82

With the currently present function (also below), the given test above takes about 400ms for me.

    pub fn get(&self, name: impl AsRef<str>) -> Option<&Atom> {
        let candidates = self
            .data
            .iter()
            .filter_map(|(ident, val)| {
                if ident.ident == name.as_ref() {
                    if true {
                        //let Source::Regular { .. } = ident.source {
                        Some((ident.source, val))
                    } else {
                        None
                    }
                } else {
                    None
                }
            })
            .collect::<Vec<_>>();

        candidates
            .into_iter()
            .max_by_key(|(source, _)| source.layer())
            .map(|(_, val)| val)
    }

It would seem logical now that removing the needless .collect() and .into_iter() should improve runtime (or at least not change it, if the compiler can optimize the allocation out). Instead, the simpler function causes the test to take around 600ms now (50% longer):

    pub fn get(&self, name: impl AsRef<str>) -> Option<&Atom> {
        self
            .data
            .iter()
            .filter_map(|(ident, val)| {
                if ident.ident == name.as_ref() {
                    if true {
                        //let Source::Regular { .. } = ident.source {
                        Some((ident.source, val))
                    } else {
                        None
                    }
                } else {
                    None
                }
            }
            .max_by_key(|(source, _)| source.layer())
            .map(|(_, val)| val)
    }

@rustbot label C-optimization I-slow E-needs-mcve

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproduction commit and regulus/src/state.rs lines 60-82, then run cargo run --release -- tests/programs/sorting_tests.re to compare the collected and direct iterator versions. Reduce the example to a minimal reproducer or benchmark that preserves the timing difference; done means the cause is isolated and the performance behavior is explained or corrected.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
compilers, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.