Change Request: Support `languageOptions.parser` to improve parser performance by integrating a Rust parser
- Dominant language
- JavaScript
- Stars
- 581
- Forks
- 92
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 11
Description
## Environment
ESLint version: HEAD
@eslint/markdown version: HEAD
Node version: 24.18.0
npm version: 11.16.0
Operating System: Windows 11
## What problem do you want to solve?
This idea came from the [Discord team channel](https://discord.com/channels/688543509199716507/688770853588172860/1519556220917252146), and this issue suggests adding `languageOptions.parser` to improve parser performance by integrating the Rust-based parser `markdown-rs`. Most of the threads here are copied from that discussion.
### Motivation
While working on Markdown plugin, I found that Markdown parsing often accounts for more than 75% of the total runtime. I tried a Rust-based prototype, and the results looked promising. For example, when the total runtime was around 100ms, Markdown parsing accounted for roughly 75ms. With the Rust prototype, parsing was reduced to around 15ms, bringing the total runtime down to around 40ms, which is roughly 2.5x faster.
Currently, `@eslint/markdown` depends on [`mdast-util-from-markdown`](https://github.com/eslint/markdown/blob/main/package.json#L104) from the [mdast](https://github.com/syntax-tree/mdast) ecosystem for Markdown parsing.
The mdast ecosystem has a Rust sibling project to `mdast-util-from-markdown` called [`markdown-rs`](https://github.com/wooorm/markdown-rs).
`markdown-rs` also provides [`to_mdast`](https://docs.rs/markdown/latest/markdown/fn.to_mdast.html), which behaves the same way as [`mdast-util-from-markdown`](https://github.com/eslint/markdown/blob/main/package.json#L104).
Frontmatter, math, and GFM parsing extensions are all supported, and based on my current experimentation, I expect most of our implemented cases would pass, since `markdown-rs` aims to produce the same AST as its JS sibling, [`mdast-util-from-markdown`](https://github.com/eslint/markdown/blob/main/package.json#L104).
### Then, how would you integrate Rust with this plugin?
If we adopt `markdown-rs`, the next challenge would be distributing cross-platform binaries for different OS and architectures (possibly including WASM binaries).
There’s currently a project called [`napi-rs`](https://napi.rs/), which bridges OS/platform-specific binaries using [N-API](https://nodejs.org/api/n-api.html).
`napi-rs` is widely used by Rust–JS projects such as [SWC](https://swc.rs/) and [Rollup](https://rollupjs.org/), so I think it would be sufficient for our use case.
I plan to create an `@eslint-markdown/parser` package (similar to `@typescript-eslint/parser`) on my end using the approach above, and I’ll evaluate how seamlessly it works with the markdown plugin as I implement this feature.
### Rust-to-JavaScript transformation algorithm?
This is the most basic approach, and there may be more opportunities to improve performance.
The Rust to JS transformation algorithm overview is:
1. Pass the Markdown UTF-8 string to Rust `markdown-rs`’s `to_mdast` parser.
2. `markdown-rs`’s `to_mdast` parser parses it at native binary speed.
3. Once parsed, stringify the AST as JSON on Rust side.
4. On the JavaScript side, receive the stringified JSON from Rust and parse it with `JSON.parse`.
5. After `JSON.parse`, we get the parsed AST on the JS side.
## Q & A
### Q. I think our parse method is still synchronous, so how would we call out to Rust?
I would keep the JS API synchronous. The idea is not to spawn a Rust executable, but to compile a small Rust wrapper around `markdown-rs` as a Node native addon using [Node-API](https://nodejs.org/api/n-api.html#writing-addons-in-various-programming-languages). Node.js loads native addons as `.node` dynamic libraries via `require()`, so the JS side can synchronously call an exported function just like it calls [`fromMarkdown()`](https://github.com/eslint/markdown/blob/main/src/language/markdown-language.js#L13) today.
### Q. Are you envisioning `execSync()`?
No. This would be direct native module loading through Node-API, not a child process. There would be no per-parse process-spawning overhead. The parse call would still run synchronously on the Node.js main thread unless we intentionally add an async API.
### Q. Is `markdown-rs` exposing a synchronous API through Node.js?
Not directly. `markdown-rs` is a Rust crate, not a ready-made Node package. We would need to add a small Rust binding layer in `@eslint/markdown` that calls the `markdown-rs` API and exposes the result to JavaScript through Node-API, likely using tooling such as [`napi-rs`](https://napi.rs/).
### Q. Is this idea stable and ready to be integrated directly into production?
Not really. This idea is still in early development and is experimental, so it may go through several iterations to evaluate how well it works in production.
## What do you think is the correct solution?
Adds `languageOptions.parser` to allow this plugin to work with a Rust-based parser and other possible parsers in the future, similar to how we use custom parsers in JS.
## Future Plan
- [x] Create and publish the experimental `@eslint-markdown/parser` package under the `canary` release tag: https://github.com/lumirlumir/npm-eslint-markdown/pull/638
- [ ] Add `languageOptions.parser` to this plugin and test whether it works seamlessly: https://github.com/eslint/markdown/pull/706
- [x] Finish the RFC: https://github.com/eslint/rfcs/pull/152
- [ ] Finish the `languageOptions.parser` implementation: https://github.com/eslint/markdown/pull/706
- [ ] Finish the parser implementation: https://github.com/lumirlumir/npm-eslint-markdown/pull/638
- [ ] Release `@eslint/markdown` with `languageOptions.parser`.
- [ ] Announce.
## Participation
- [x] I am willing to submit a pull request for this change.
## AI acknowledgment
- [x] I did not use AI to generate this issue report.
- [ ] (If the above is not checked) I have reviewed the AI-generated content before submitting.
## Additional comments
- Per the discussion in Discord, `languageOptions.parser` would apply only to the Markdown parser and may not be generalizable to all language plugins.
Contributor guide
Assessment
This issue has not been assessed yet.