microsoft / microsoft/language-server-protocol
Clarify purpose of LSIF item edge 'shard' or remove?
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 13k
- Forks
- 1k
- Avg merge
- 6d 1h
- Merged PRs (30d)
- 10
Description
LSIF item edges contain a "shard" property referenced in https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_definition
The spec has this to say about it:
The item edge as an additional property shard which indicate the vertex that is the source (e.g. a document or a project) of these declarations. We added this information to still make it easy to emit the data but also make it easy to process and shard the data when storing into a database. Without that information we would either need to specific an order in which data needs to be emitted (e.g. an item edge and only refer to a range that got already added to a document using a contains edge) or we force processing tools to keep a lot of vertices and edges in memory. The approach of having this shard property looks like a fair balance.
I read into the 'shard' property when doing the performance investigation that prompted #1755. Based on the description, shard seems to provide a means to partition the output of an LSIF file. My hope was that this could be used to parallelize or batch processing of the LSIF, but my updated understanding is that is actually just the equivalent of a partition key, much as one would use to partition a database.
Can the spec be updated to indicate a sample usage (DB partition key? output file clustering?) and the limitations of the shard (does not enable parallel parsing or splitting into subgraphs).
Alternatively, is this a candidate for removal/simplification?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the LSIF item-edge shard description in the linked LSP 3.17 specification and review the performance investigation in #1755. Compare the stated purpose with the observed limitations, then update the specification with a concrete usage and limitations or determine whether the property should be removed or simplified.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100