expert-lsp / expert-lsp/expert

Possible to cache the index/state for big projects?

Open
#72 4 comments 16 reactions 0 assignees View on GitHub
research
Dominant language
Elixir
Stars
2.1k
Forks
113
Avg merge
1d 10h
Merged PRs (30d)
26

Description

Let me start by thanking the great work you've been doing 🌟

From my tests it has been working super well for "regular-sized" projects.

At work we have a big monolith with more than ~3M lines of Elixir code, and all the language servers we tried more or less worked but:
1. It took a lot of time to index everything, which meant developers had to wait some time before they could use the LSP tools.
* This also used a lot of CPU and RAM, I assume due to the full build the language server needs to do
2. The LSP tools stopped working after a while, usually after some `pulls` from `master` when a lot of changes were pulled (we have >100 developers working on the same code base).

To improve the developer experience for these big projects, would it be possible to cache the state/index of the language server, so that when pulling a given version of the project we would also download the corresponding Expert index and all the LSP tools would work immediately?

For context, with a 32 cores desktop + 64GB ram it took ~45minutes for Expert to finish indexing (I've monitored `btop` and all the cores being used during indexing, until they became mostly idle); folks with Mac laptops are mentioning letting the index run for 3-4 hours before it was usable.

Happy to provide more detailed info/timings; is there a way to collect this kind of telemetry from Expert?

If you already have an idea on how this caching mechanism could fit in the Expert architecture I can try to take a stab at it.

Once more, super thank you for this 🙏

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.