microsoft / microsoft/language-server-protocol

Clarify purpose of LSIF item edge 'shard' or remove?

Open
#1,756 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

clarification lsif
Dominant language
TypeScript
Stars
13k
Forks
1k
Avg merge
6d 1h
Merged PRs (30d)
10

Description

LSIF item edges contain a "shard" property referenced in https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_definition

The spec has this to say about it:

The item edge as an additional property shard which indicate the vertex that is the source (e.g. a document or a project) of these declarations. We added this information to still make it easy to emit the data but also make it easy to process and shard the data when storing into a database. Without that information we would either need to specific an order in which data needs to be emitted (e.g. an item edge and only refer to a range that got already added to a document using a contains edge) or we force processing tools to keep a lot of vertices and edges in memory. The approach of having this shard property looks like a fair balance.

I read into the 'shard' property when doing the performance investigation that prompted #1755. Based on the description, shard seems to provide a means to partition the output of an LSIF file. My hope was that this could be used to parallelize or batch processing of the LSIF, but my updated understanding is that is actually just the equivalent of a partition key, much as one would use to partition a database.

Can the spec be updated to indicate a sample usage (DB partition key? output file clustering?) and the limitations of the shard (does not enable parallel parsing or splitting into subgraphs).

Alternatively, is this a candidate for removal/simplification?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the LSIF item-edge shard description in the linked LSP 3.17 specification and review the performance investigation in #1755. Compare the stated purpose with the observed limitations, then update the specification with a concrete usage and limitations or determine whether the property should be removed or simplified.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.