redhat-developer / redhat-developer/lsp4ij

Add support for custom semantic token types and modifiers

Open
#871 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement semantic tokens user-defined LS
Dominant language
Java
Stars
344
Forks
113
Avg merge
5h 22m
Merged PRs (30d)
15

Description

We've discussed this a few times, but some language servers return semantic token types that aren't present in SemanticTokenTypes and/or semantic token modifiers that aren't present in SemanticTokenModifiers. Sometimes those custom token types/modifiers are actually quite important, e.g., member from the TypeScript language server, and if not understood explicitly by LSP4IJ, important functionality can be missing ranging from semantic syntax highlighting to the way that a given token should be translated into an element the PSI hierarchy.

It should be possible for language server definitions to augment the known semantic token types and modifiers via both code and configuration. One way to do this would be to add the following methods to LSPSemanticTokensFeature:

/**
 * Returns custom semantic token types for the file.
 *
 * @param file the file
 * @return the file's custom semantic token types
 */
@NotNull
public Set<LSPSemanticTokenType> getCustomSemanticTokenTypes(@NotNull PsiFile file) {
    return Collections.emptySet();
}

/**
 * Returns custom semantic token modifiers for the file.
 *
 * @param file the file
 * @return the file's custom semantic token modifiers
 */
@NotNull
public Set<LSPSemanticTokenModifier> getCustomSemanticTokenModifiers(@NotNull PsiFile file) {
    return Collections.emptySet();
}

where LSPSemanticTokenType is defined as:

/**
 * A custom semantic token type.
 */
public class LSPSemanticTokenType {
    /** The semantic token type name */
    public String name;
    /** Whether or not the semantic token should be interpreted as being an identifier */
    public boolean identifier = false;
    /** Whether or not the semantic token should be interpreted as being a type */
    public boolean type = false;
    /** Whether or not the semantic token should be interpreted as being a keyword */
    public boolean keyword = false;
    /** The name of the existing semantic token type from which this one should inherit its behavior */
    public String inheritFrom = null;
    /** The text attributes key that should be used for syntax highlighting of this semantic token */
    public String textAttributesKey = null;
}

and LSPSemanticTokenModifier is defined as:

/**
 * A custom semantic token modifier.
 */
public class LSPSemanticTokenModifier {
    /** The semantic token modifier name */
    public String name = null;
    /** Whether or not semantic tokens with this modifier should be interpreted as being a declaration */
    public boolean declaration = false;
    /** The name of the existing semantic token modifier from which this one should inherit its behavior */
    public String inheritFrom = null;
    /** The text attributes key that should be used for syntax highlighting of semantic tokens with this modifier */
    public String textAttributesKey = null;
}

Custom language server definitions would just override these methods to return any known semantic token types/modifiers returned by the associated language server. User-defined language server definitions would provide the same information via client configuration, e.g.:

{
  "semanticTokens": {
    "customTokens": {
      "label": {
        "identifier": true,
        "textAttributesKey": "LSP_LABEL"
      }
      "member": {
        "identifier": true,
        "textAttributesKey": "LSP_METHOD"
      }
    },
    "customModifiers": {
      "virtual": {
        "inheritFrom": "abstract"
      }
    }
  }
}

DefaultSemanticTokensColorProvider would then be updated to use this information if available, as would LSPSemanticTokensFileViewProviderHelper that is being added in #853.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading LSPSemanticTokensFeature, DefaultSemanticTokensColorProvider, and LSPSemanticTokensFileViewProviderHelper, including the related work in #853. Trace how built-in token types and modifiers reach syntax highlighting and PSI handling, then define completion as custom values working through both language-server code and client configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
developer-experience, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.