Nimblesite / Nimblesite/Basilisk
No latency contract for LSP requests: inlay hints/completion run per keystroke with no target and no ratchet
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 54
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Summary
There is no latency contract for any LSP request, and no gate that would catch a
regression in one. [LSPARCH-FEATURES-INLAYHINTS] is two sentences and says
nothing about cost:
Hints cover inferred variable/return types, generic parameters, and call-site
parameter names. Configuration gates are applied before results are returned.
The benchmark ratchet in CLAUDE.md / [CHKARCH-TESTING-BENCH-RATCHET] governs
CLI check wall-clock only. Nothing measures textDocument/inlayHint,
textDocument/completion, hover, or semantic tokens, so their latency can move
in either direction without any signal.
This is filed as a missing contract, not as a performance bug — there is no
target to have regressed against.
What the numbers currently are
Measured against basilisk lsp over stdio on
JelleZijlstra/taxonomy @9b359fa,
taxonomy/shell.py (3,550 lines), sending a didChange before each request so
per-request work cannot be amortised:
| Request | median | p90 | max | result size |
|---|---|---|---|---|
textDocument/inlayHint (full file) |
25.0 ms | 31.3 ms | 49.2 ms | 136 hints |
textDocument/completion ("abc".) |
11.6 ms | 14.5 ms | 28.3 ms | 65 items |
A real VS Code session on the same file recorded 90 ms for a single
inlayHint over a 72-line range, concurrent with a diagnostics publish — so
under genuine editor contention the cost is higher than an idle synthetic client
suggests.
Both requests fire on every keystroke. Neither figure is alarming on its own;
the point is that nothing pins them.
Why it matters now
Two recent changes moved per-request cost with no measurement either way:
- #377 made
textDocument/completionrunresolve_module_importson every
request (crates/basilisk-lsp/src/server/handlers/features.rs). Builtins are
cached per snapshot, but the call is per-request. - Inlay hints recompute a whole-file pass per keystroke, and (per the companion
issue) throw the entire result away whenever the buffer does not parse.
Either could regress 5× and no test, ratchet, or CI job would notice.
Proposal
- State a target per interactive request in
[LSPARCH-FEATURES-*]— a
budget of the order of one input frame (~16 ms) for hints/completion on a file
of a stated size, or whatever the team judges right. The number matters less
than its existence. - Measure it in the harness that already exists. The WebSocket E2E fixture
(crates/basilisk-lsp/tests/lsp/ws_test_common.rs) already drives real
JSON-RPC; timing an assertion there is a small addition. - Ratchet it the same way
[CHKARCH-TESTING-BENCH-RATCHET]ratchets CLI
times — read the committed baseline, fail on regression.
Worth deciding explicitly as part of this: whether the target is per-request
wall-clock, or per-keystroke total across the requests an edit triggers
(didChange currently fans out to inlayHint, codeLens, foldingRange, and
diagnostics).
Environment
basilisk 0.0.0-PLACEHOLDER, cleancargo build --releaseofmain@e3e97d30- macOS 15 (darwin 25.5.0), arm64, otherwise idle for the synthetic runs
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the proposal and the existing benchmark ratchet in CLAUDE.md, then inspect crates/basilisk-lsp/tests/lsp/ws_test_common.rs and the request paths in crates/basilisk-lsp/src/server/handlers/features.rs. Decide the per-request or per-keystroke target with maintainers, add timing coverage, and make the committed baseline fail on regression.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- devtools, performance, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100