[Bug] monaco.editor.colorize ignores DocumentSemanticTokensProvider

Reproducible in vscode.dev or in VS Code Desktop?

• Not reproducible in vscode.dev or VS Code Desktop

Reproducible in the monaco editor playground?

• Not reproducible in the monaco editor playground

Monaco Editor Playground Link

The monaco.editor.colorize ignores DocumentSemanticTokensProvider.

I'm not sure if I don't understand its functionality but for me it seems to be a bug since it doesn't return the code colorized with DocumentSemanticTokensProvider.

In the example bellow, check the output in console. It's <span><span class="mtk1">[class.static.token]  [type.static.abstract]</span></span><br/>. As you may notice, all returned colorized code are with the same color , while in editor class.static.token and type.static.abstract are red and green.

Link to the Monaco Editor Playground with the example.

Monaco Editor Playground Code

Just use the example `Semantic Tokens Provider Example` and paste the following code at the end:

`monaco.editor.colorize("[class.static.token]  [type.static.abstract]", "plaintext", {}).then((code) => {
    console.log(code)
});`

Check the console.

Reproduction Steps

1. Set a DocumentSemanticTokensProvider using registerDocumentSemanticTokensProvider for your language
2. Call monaco.editor.colorize to colorize a piece of code for your language
3. The returned result ignores the semantic

Actual (Problematic) Behavior

No response

Expected Behavior

The returned piece of code passed to monaco.editor.colorize should consider the semantic set in registerDocumentSemanticTokensProvider

Additional Context

No response

Start with the linked Monaco Editor Playground and the Semantic Tokens Provider Example, then inspect the interaction between registerDocumentSemanticTokensProvider and monaco.editor.colorize. Reproduce the console output and compare it with the editor rendering; done means colorize returns markup that reflects the registered semantic tokens.