Consider adding per-request scratch space?

Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer/topic/Per.20request.20scratch.20spaces.20.28via.20bumpallo.29

See https://lobste.rs/s/afqbdk/individual_element_thinking_vs_grouped and https://news.ycombinator.com/item?id=26938367.

The TL;DR is that, when you have a request-response style thing, it makes sense to implement the "scratch space pattern". At the start of the cycle, you create a bump allocator and pass it throughout the stack. Things stored in this allocator are !Drop, so they don't create drop glue. Instead, at the very end, once we send the response down the tube, the scratch space is cleaned up in one call.

In terms of rust-analyzer's API, that means that we introduce a ScratchSpace struct and thread it through methods:

// ide/lib.rs

#[derive(Default)]
pub struct  ScratchAlloc {
  b: bumpallo::Bump
}

impl Analysis {
    /// Computes syntax highlighting for the given file
    pub fn highlight(&self, a: &ScratchAlloc, file_id: FileId) -> Cancelable<Vec<HlRange>> {
        self.with_db(|db| syntax_highlighting::highlight(db, file_id, None, false))
    }
}

After this initial threading, we can take advantage of scratch space internally, to allocate temporary things. We might go even further, and allocate the return value in the scratch space:

impl Analysis {
    /// Computes syntax highlighting for the given file
    pub fn highlight(&self, a: &'a ScratchAlloc, file_id: FileId) -> Cancelable<scratch::Vec<'a, HlRange<'a>>> {
        self.with_db(|db| syntax_highlighting::highlight(db, file_id, None, false))
    }
}

Note that this latter step changes the API in an unfortunate way, as now the return value is not owned, so it becomes harder for callee to use.

I am torn on whether we should actually do steps 1&2. On the one hand, I feel pretty strongly that not using scratch spaces is sloppiness, which leaves a bunch of perf on the table. It also seems that this is one of those cross-cutting concerns that you really should start with as early as possible, as retro-fitting it would be hard. On the other hand, I also feel strongly that, at this stage, practically, this won't be any meaningful win for us. So, 🤷

As a side note, I think scratch allocators are fundamentally different from the custom allocators being designed. The fundamental thing we are aiming at here is "no drop glue", which I think isn't really possible with just custom allocators.

Start with ide/lib.rs and the Analysis::highlight example, then read the linked Zulip discussion about per-request scratch spaces. Assess whether introducing ScratchAlloc and threading it through the API would provide a meaningful win and how the proposed return-value change affects callers. Done requires a clear decision on whether steps 1 and 2 should be pursued.