Suggestion: Extensibility API for custom language modes within an HTML-like document

[rbuckton]

I've been working on and off for a few years on a VS Code extension for Ecmarkup, an HTML dialect used for writing programming language specifications (specifically, ECMAScript/JavaScript and TypeScript). I had initially tried to leverage the html-language-service directly for this, but it fell short of my needs. In the end I started to build a more flexible service for handling mixed-language documents so that I could have an improved "find-all-references" and "go-to-definition" experience.

When the ability to supply custom tag data to the HTML language service became available, I was hoping to investigate leveraging that, but unfortunately it still falls short of my needs. What I'd like to be able to do is extend the vscode-html-languageservice to include three additional language modes, based on specific tags/attributes in the document:

1. Ecmarkup uses a <pre class="metadata"> tag to include YAML front-matter in the document that is stripped during processing.
2. Ecmarkup uses a markdown-like syntax for defining algorithm steps in an <emu-alg> tag using Ecmarkdown. Ecmarkdown content can also be parsed in the content of any HTML element.
3. Ecmarkup uses a specialized syntax for defining syntactic grammar in an <emu-grammar> tag using Grammarkdown.

My goals are:

• Support YAML, Ecmarkdown, and Grammarkdown as separate language modes inside of an Ecmarkup HTML document.
• Leverage the existing language services features for Grammarkdown (currently published as grammarkdown-vscode) to parse <emu-grammar> content. This includes existing capabilities for "find-all-references", "go-to-definition", and others for Grammarkdown content.
• Use all of the collective information to support "find-all-references" and "go-to-definition" across the entire document. For example, a grammar production such as MemberExpression might be defined in an <emu-grammar> clause, and later referenced elsewhere within the specification using Ecmarkdown syntax:

  <emu-clause id="...">
    <h1>Syntax</h1>
    <emu-grammar>
      MemberExpression : 
        LeftHandSideExpression
        MemberExpression `.` IdentifierName
        MemberExpression `[` Expression `]`
    </emu-grammar>
    <p>
      A |MemberExpression| is ...
    </p>
  </emu-clause>

Currently all of this requires a custom language server that emulates the mixed-language parsing features already present in vscode-html-languageservice, with the added complexity that Grammarkdown is a whitespace-sensitive dialect (and the current vscode-html-languageservice behavior or just replacing non-mode-specific text content with whitespace can cause problems with Grammarkdown's language features.

As a result, my asks are:

• Allow for the ability for another extension to wrap the vscode-html-languageservice
• Modify vscode-html-languageservcie to provide extensibility points to hook additional custom parsers and add additional language modes.

Related: microsoft/vscode-custom-data#16, microsoft/vscode-custom-data#17

[aeschli]

Wrapping the vscode-html-languageservice is what we already do in our HTML language server. It's how we bring in the support for embedded JavaScript and CSS.
https://code.visualstudio.com/api/language-extensions/embedded-languages#language-services describes this in detail.

The model is that vscode-html-languageservice is a library that focuses on HTML but can be used as a building block for a richer service or server.

Maybe your request is to extend the html language server to support more embedded languages, or is it really the service?

For the service we have limited participation capability such as the completion participant. Maybe that's what you have in mind.
The challenge for such participant is to come up with an API that is rich enough.

[aeschli]

Not sure what API additions you have in mind. Please list the API's that you are missing.