facebook / facebook/docusaurus

MagicComments should be applied after Prism tokenization to properly track line numbers

Đang mở
#8,550 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
bug
Ngôn ngữ chính
TypeScript
Star
66.2k
Fork
10k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
52

Mô tả

### Have you read the Contributing Guidelines on issues?

- [X] I have read the [Contributing Guidelines on issues](https://github.com/facebook/docusaurus/blob/main/CONTRIBUTING.md#reporting-new-issues).

### Prerequisites

- [X] I'm using the latest version of Docusaurus.
- [ ] I have tried the `npm run clear` or `yarn clear` command.
- [ ] I have tried `rm -rf node_modules yarn.lock package-lock.json` and re-installing packages.
- [X] I have tried creating a repro with https://new.docusaurus.io.
- [ ] I have read the console error message carefully (if applicable).

### Description

The way magic comments are currently [applied](https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-theme-classic/src/theme/CodeBlock/Content/String.tsx#L48) (before passing the code to Prism) interferes with Prism `after-tokenize` hook (and possibly other hooks too). I implemented a way to manually specify type names for proper highlighting:

image

(this is a screenshot from the webpage)

This requires me to erase each line with a specific pattern:

```cpp
// custom-syntactic-tweak-that-adds-types:Vehicle
```

This results in magic comments being shifted.

### Reproducible demo

https://codesandbox.io/s/docusaurus-magiccomments-breaks-lines-n78f0r?file=/docusaurus.config.js

### Steps to reproduce

To see the problem:
1. Click on "Tutorial" to go to the intro doc.

How to reproduce it starting from a fresh installation:
1. Using `prism-include-languages.js` hook to Prism `after-tokenize` and erase any token that contains a new line
2. Test how `// highlight-next-line` works in a code block

### Expected behavior

Magic comments are applied after the `after-tokenize` hook so that user-provided hooks work if lines are added or removed.

### Actual behavior

Magic comments are applied before passing the code to the Prism tokenizer and this results in highlighted lines being shifted if a hook is used to add/remove tokens that contains new lines.

### Your environment

- Public source code: https://codesandbox.io/s/docusaurus-magiccomments-breaks-lines-n78f0r?file=/docusaurus.config.js
- Public site URL: https://n78f0r.sse.codesandbox.io/docs/intro
- Docusaurus version used: 2.2.0
- Environment name and version (e.g. Chrome 89, Node.js 16.4): Brave v1.47.171 (Jan 12, 2023), Node.js v16.15.1
- Operating system and version (e.g. Ubuntu 20.04.2 LTS): CodeSandbox (?)

### Self-service

- [ ] I'd be willing to fix this bug myself.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.