Suggestion: Extensibility API for custom language modes within an HTML-like document

I've been working on and off for a few years on a VS Code extension for Ecmarkup, an HTML dialect used for writing programming language specifications (specifically, ECMAScript/JavaScript and TypeScript). I had initially tried to leverage the html-language-service directly for this, but it fell short of my needs. In the end I started to build a more flexible service for handling mixed-language documents so that I could have an improved "find-all-references" and "go-to-definition" experience.

When the ability to supply custom tag data to the HTML language service became available, I was hoping to investigate leveraging that, but unfortunately it still falls short of my needs. What I'd like to be able to do is extend the vscode-html-languageservice to include three additional language modes, based on specific tags/attributes in the document:

1. Ecmarkup uses a <pre class="metadata"> tag to include YAML front-matter in the document that is stripped during processing.
2. Ecmarkup uses a markdown-like syntax for defining algorithm steps in an <emu-alg> tag using Ecmarkdown. Ecmarkdown content can also be parsed in the content of any HTML element.
3. Ecmarkup uses a specialized syntax for defining syntactic grammar in an <emu-grammar> tag using Grammarkdown.

My goals are:

• Support YAML, Ecmarkdown, and Grammarkdown as separate language modes inside of an Ecmarkup HTML document.
• Leverage the existing language services features for Grammarkdown (currently published as grammarkdown-vscode) to parse <emu-grammar> content. This includes existing capabilities for "find-all-references", "go-to-definition", and others for Grammarkdown content.
• Use all of the collective information to support "find-all-references" and "go-to-definition" across the entire document. For example, a grammar production such as MemberExpression might be defined in an <emu-grammar> clause, and later referenced elsewhere within the specification using Ecmarkdown syntax:

  <emu-clause id="...">
    <h1>Syntax</h1>
    <emu-grammar>
      MemberExpression : 
        LeftHandSideExpression
        MemberExpression `.` IdentifierName
        MemberExpression `[` Expression `]`
    </emu-grammar>
    <p>
      A |MemberExpression| is ...
    </p>
  </emu-clause>
  

Currently all of this requires a custom language server that emulates the mixed-language parsing features already present in vscode-html-languageservice, with the added complexity that Grammarkdown is a whitespace-sensitive dialect (and the current vscode-html-languageservice behavior or just replacing non-mode-specific text content with whitespace can cause problems with Grammarkdown's language features.

As a result, my asks are:

• Allow for the ability for another extension to wrap the vscode-html-languageservice
• Modify vscode-html-languageservcie to provide extensibility points to hook additional custom parsers and add additional language modes.

Related: https://github.com/microsoft/vscode-custom-data/issues/16, https://github.com/microsoft/vscode-custom-data/issues/17