rust-lang / rust-lang/rust-analyzer

Possible Linux(glibc) memory usage improvement

Open
#19,174 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
16.9k
Forks
2.2k
Avg merge
1d 12h
Merged PRs (30d)
72

Description

So I was watching this video and wondered if Rust-Analyzer could take this and use it to improve it's memory usage.

It tells GLIBC to not pool large allocations in arenas and can prevent a pile-up of wasted memory space.

I did take the code from cosmic-bg and just added the call to mallopt(M_MMAP_THRESHOLD, 65536); and saw a memory usage decrease. Only like a 100mb however, I think this might want to be explored.

The Cloudflare article mentions calling malloc_trim from time to time as this goes through and removes unused arena space. Which I think might be a good idea to call after project loads?

Code

#[cfg(target_env = "gnu")]
mod malloc {
    use std::os::raw::c_int;
    const M_MMAP_THRESHOLD: c_int = -3;

    extern "C" {
        fn mallopt(param: c_int, value: c_int) -> c_int;
    }
	/// tells GLIBC to not pool large allocations in arenas  
    pub(crate) fn limit_mmap_threshold() {
        unsafe {
            mallopt(M_MMAP_THRESHOLD, 65536);
        }
    }
}
fn main() {
	#[cfg(target_env = "gnu")]
    malloc::limit_mmap_threshold();
    // The rest of the main function
}

The reason this is an issue and not a pull request is I think maybe more experimentation might be needed first and also want to give the developers with more knowledge a chance to give any thoughts on this subject.

Links

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from rust-analyzer's main entry point and review how Linux glibc allocations behave during project loads. Experiment with the proposed mallopt and malloc_trim approaches, measuring memory use and checking their platform and safety implications; done would require evidence of a reliable improvement and agreement on the approach.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust
Domain
devtools, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.