facebook / facebook/docusaurus

MagicComments should be applied after Prism tokenization to properly track line numbers

Abierto
#8,550 1 comentario 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
TypeScript
Estrellas
66.2k
Forks
10k
Merge medio
1 d 3 h
PR fusionados (30 d)
52

Descripción

### Have you read the Contributing Guidelines on issues?

- [X] I have read the [Contributing Guidelines on issues](https://github.com/facebook/docusaurus/blob/main/CONTRIBUTING.md#reporting-new-issues).

### Prerequisites

- [X] I'm using the latest version of Docusaurus.
- [ ] I have tried the `npm run clear` or `yarn clear` command.
- [ ] I have tried `rm -rf node_modules yarn.lock package-lock.json` and re-installing packages.
- [X] I have tried creating a repro with https://new.docusaurus.io.
- [ ] I have read the console error message carefully (if applicable).

### Description

The way magic comments are currently [applied](https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-theme-classic/src/theme/CodeBlock/Content/String.tsx#L48) (before passing the code to Prism) interferes with Prism `after-tokenize` hook (and possibly other hooks too). I implemented a way to manually specify type names for proper highlighting:

image

(this is a screenshot from the webpage)

This requires me to erase each line with a specific pattern:

```cpp
// custom-syntactic-tweak-that-adds-types:Vehicle
```

This results in magic comments being shifted.

### Reproducible demo

https://codesandbox.io/s/docusaurus-magiccomments-breaks-lines-n78f0r?file=/docusaurus.config.js

### Steps to reproduce

To see the problem:
1. Click on "Tutorial" to go to the intro doc.

How to reproduce it starting from a fresh installation:
1. Using `prism-include-languages.js` hook to Prism `after-tokenize` and erase any token that contains a new line
2. Test how `// highlight-next-line` works in a code block

### Expected behavior

Magic comments are applied after the `after-tokenize` hook so that user-provided hooks work if lines are added or removed.

### Actual behavior

Magic comments are applied before passing the code to the Prism tokenizer and this results in highlighted lines being shifted if a hook is used to add/remove tokens that contains new lines.

### Your environment

- Public source code: https://codesandbox.io/s/docusaurus-magiccomments-breaks-lines-n78f0r?file=/docusaurus.config.js
- Public site URL: https://n78f0r.sse.codesandbox.io/docs/intro
- Docusaurus version used: 2.2.0
- Environment name and version (e.g. Chrome 89, Node.js 16.4): Brave v1.47.171 (Jan 12, 2023), Node.js v16.15.1
- Operating system and version (e.g. Ubuntu 20.04.2 LTS): CodeSandbox (?)

### Self-service

- [ ] I'd be willing to fix this bug myself.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.