facebook / facebook/docusaurus
MagicComments should be applied after Prism tokenization to properly track line numbers
- Vorherrschende Sprache
- TypeScript
- Sterne
- 66.2k
- Forks
- 10k
- Ø Merge
- 1 T. 3 Std.
- Gemergte PRs (30 T.)
- 52
Beschreibung
### Have you read the Contributing Guidelines on issues?
- [X] I have read the [Contributing Guidelines on issues](https://github.com/facebook/docusaurus/blob/main/CONTRIBUTING.md#reporting-new-issues).
### Prerequisites
- [X] I'm using the latest version of Docusaurus.
- [ ] I have tried the `npm run clear` or `yarn clear` command.
- [ ] I have tried `rm -rf node_modules yarn.lock package-lock.json` and re-installing packages.
- [X] I have tried creating a repro with https://new.docusaurus.io.
- [ ] I have read the console error message carefully (if applicable).
### Description
The way magic comments are currently [applied](https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-theme-classic/src/theme/CodeBlock/Content/String.tsx#L48) (before passing the code to Prism) interferes with Prism `after-tokenize` hook (and possibly other hooks too). I implemented a way to manually specify type names for proper highlighting:

(this is a screenshot from the webpage)
This requires me to erase each line with a specific pattern:
```cpp
// custom-syntactic-tweak-that-adds-types:Vehicle
```
This results in magic comments being shifted.
### Reproducible demo
https://codesandbox.io/s/docusaurus-magiccomments-breaks-lines-n78f0r?file=/docusaurus.config.js
### Steps to reproduce
To see the problem:
1. Click on "Tutorial" to go to the intro doc.
How to reproduce it starting from a fresh installation:
1. Using `prism-include-languages.js` hook to Prism `after-tokenize` and erase any token that contains a new line
2. Test how `// highlight-next-line` works in a code block
### Expected behavior
Magic comments are applied after the `after-tokenize` hook so that user-provided hooks work if lines are added or removed.
### Actual behavior
Magic comments are applied before passing the code to the Prism tokenizer and this results in highlighted lines being shifted if a hook is used to add/remove tokens that contains new lines.
### Your environment
- Public source code: https://codesandbox.io/s/docusaurus-magiccomments-breaks-lines-n78f0r?file=/docusaurus.config.js
- Public site URL: https://n78f0r.sse.codesandbox.io/docs/intro
- Docusaurus version used: 2.2.0
- Environment name and version (e.g. Chrome 89, Node.js 16.4): Brave v1.47.171 (Jan 12, 2023), Node.js v16.15.1
- Operating system and version (e.g. Ubuntu 20.04.2 LTS): CodeSandbox (?)
### Self-service
- [ ] I'd be willing to fix this bug myself.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.