facebook / facebook/docusaurus

MagicComments should be applied after Prism tokenization to properly track line numbers

Ouverte
#8,550 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
bug
Langage dominant
TypeScript
Étoiles
66.2k
Forks
10k
Merge moyen
1 j 3 h
PR mergées (30 j)
52

Description

### Have you read the Contributing Guidelines on issues?

- [X] I have read the [Contributing Guidelines on issues](https://github.com/facebook/docusaurus/blob/main/CONTRIBUTING.md#reporting-new-issues).

### Prerequisites

- [X] I'm using the latest version of Docusaurus.
- [ ] I have tried the `npm run clear` or `yarn clear` command.
- [ ] I have tried `rm -rf node_modules yarn.lock package-lock.json` and re-installing packages.
- [X] I have tried creating a repro with https://new.docusaurus.io.
- [ ] I have read the console error message carefully (if applicable).

### Description

The way magic comments are currently [applied](https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-theme-classic/src/theme/CodeBlock/Content/String.tsx#L48) (before passing the code to Prism) interferes with Prism `after-tokenize` hook (and possibly other hooks too). I implemented a way to manually specify type names for proper highlighting:

image

(this is a screenshot from the webpage)

This requires me to erase each line with a specific pattern:

```cpp
// custom-syntactic-tweak-that-adds-types:Vehicle
```

This results in magic comments being shifted.

### Reproducible demo

https://codesandbox.io/s/docusaurus-magiccomments-breaks-lines-n78f0r?file=/docusaurus.config.js

### Steps to reproduce

To see the problem:
1. Click on "Tutorial" to go to the intro doc.

How to reproduce it starting from a fresh installation:
1. Using `prism-include-languages.js` hook to Prism `after-tokenize` and erase any token that contains a new line
2. Test how `// highlight-next-line` works in a code block

### Expected behavior

Magic comments are applied after the `after-tokenize` hook so that user-provided hooks work if lines are added or removed.

### Actual behavior

Magic comments are applied before passing the code to the Prism tokenizer and this results in highlighted lines being shifted if a hook is used to add/remove tokens that contains new lines.

### Your environment

- Public source code: https://codesandbox.io/s/docusaurus-magiccomments-breaks-lines-n78f0r?file=/docusaurus.config.js
- Public site URL: https://n78f0r.sse.codesandbox.io/docs/intro
- Docusaurus version used: 2.2.0
- Environment name and version (e.g. Chrome 89, Node.js 16.4): Brave v1.47.171 (Jan 12, 2023), Node.js v16.15.1
- Operating system and version (e.g. Ubuntu 20.04.2 LTS): CodeSandbox (?)

### Self-service

- [ ] I'd be willing to fix this bug myself.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.