rokucommunity / rokucommunity/brighterscript

Improve language server performance for large multi-project workspaces

Open
#1,625 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
208
Forks
68
Avg merge
8h 39m
Merged PRs (30d)
39

Description

Context

The BrighterScript language server works well for single-project and small multi-project workspaces.

But as workspace scale increases, for example a monorepo with 30-60+ bsconfig.json files in a single VS Code workspace, certain design choices that are reasonable at smaller scale begin to show cumulative overhead, leading to significantly elevated CPU usage, out-of-memory errors, and excessive battery drain on local development machines.

This ticket captures our analysis, and a set of targeted, low-risk improvements to address this.

Root Causes Identified
  1. File-change event storms
    Every file-system event triggers onDidChangeWatchedFiles synchronously. In large workspaces, rapid successive events - e.g. from git checkout, bulk saves, or build tool output - each trigger a full processing cycle including PathFilterer rebuilds and project-level handleFileChanges calls. This means a burst of 50 events results in 50 independent processing cycles, each fanning out to all N projects
  2. Unbounded parallel project activation
    syncProjects uses Promise.all to activate every discovered project simultaneously. When 30-60+ projects are discovered, all of them parse files, resolve plugins, and validate in parallel, creating a CPU spike that can saturate all cores and leave the system unresponsive during startup.
  3. Redundant PathCollection creation
    flushDocumentChanges creates a new PathCollection instance for every project on every document flush (~150ms after each edit). Each construction compiles glob patterns into regex matchers via micromatch.matcher(). Since a project's file patterns only change on reload, this produces identical matchers every time, and redundant work that compounds with project count.
Proposed improvements

Rather than pursuing large architectural changes (lazy project loading, shared AST caches, declaration boundaries), which would be a significant long-term effort, the below 3 proposed improvements target the highest-impact multipliers within the current architecture:

  1. Debounce onDidChangeWatchedFiles
    https://github.com/rokucommunity/brighterscript/pull/1626
    Accumulate file-change events in a buffer and flush them as a single batch after a short configurable delay (e.g. 300ms). This collapses burst scenarios (e.g. git checkout generating dozens of events) into one processing cycle instead of one per event.
Before After
50 rapid events × N projects = 50N processing cycles 1 batched flush × N projects = N processing cycles
  1. Concurrency-limit project activation
    https://github.com/rokucommunity/brighterscript/pull/1627
    Replace Promise.all in syncProjects with a worker-pool pattern that limits how many projects activate concurrently. All projects still activate, but the CPU load is spread over time rather than spiking all at once, keeping the system responsive during startup.
Before After
N projects activate in parallel → CPU spike N projects activate in waves of 3 → smoother resource usage
  1. Cache PathCollection per project
    https://github.com/rokucommunity/brighterscript/pull/1628
    Use a WeakMap to lazily create and cache PathCollection instances per project. File patterns only change when a project reloads, so the cached filterer stays valid for the project's lifetime and is automatically garbage-collected when the project is removed.
Before After
New PathCollection compiled per project per flush One PathCollection per project, reused until project reloads

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the language server entry points for onDidChangeWatchedFiles, syncProjects, and flushDocumentChanges, then trace how PathCollection is created and project activation is scheduled. Compare the current behavior with the three proposed improvements and verify that event bursts, large-project startup, and repeated document flushes no longer cause redundant work while all projects still activate correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
developer-experience, devtools, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.