[Proposal] Allow embedding other languages in macros

A lot of function-like procedural macros might have syntax alien to Rust (html! Yew macro comes to mind). Highlighing based on expansion is not always reliable as a lot of tokens might not end up in the final code.

So instead of trying to handle every case, I believe there should be an official way to extend rust-analyzer with custom language services and syntax highlights, this would allow library authors to offer better DX for their custom macros.

Proposal

A TextMate scope for macros (VSCode)

A custom scope like meta.macro-function.rust would allow syntax highlighting coming from somewhere else, since scope names cannot be dynamic, the scope should span the entire macro invocation (my_macro!(...)), this way rules can be matched based on the macro's name.

Registering external macro LSPs

External LSPs could be registered for macros and rust-analyzer would forward requests to them. In VSCode, this could be done via the extension API based on this stack overflow answer.

This way rust-analyzer can decide not to forward known macros it can handle, and there are no multiple LSPs processing the same document simultaneously, potentially clashing with each other.

Macros would be processed in the following order:

• Known macros (e.g. println)
• Registered LSP if found by the macro path
• Best-effort based on expansion

Unresolved questions

• I only considered VSCode, as I don't use or know the internals of other editors rust-analyzer supports.
• Macros can be imported by a different name. Rust-analyzer could find out its original name/path, but this wouldn't work for syntax highlights.
• Scrap the TextMate highlight support, and allow macro LSPs to provide semantic tokens.
• This feature would further encourage people to write macros with custom alien syntax, but it's already happening anyway.

The proposal names no files, tests, or entry points; start by studying rust-analyzer's existing macro expansion and syntax-highlighting behavior, along with its VSCode integration. The unresolved questions about editor support, renamed macros, and semantic tokens need decisions before completion can be defined.