Change Request: Provide a TokenSourceCode
- Dominant language
- JavaScript
- Stars
- 348
- Forks
- 51
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 11
Description
### Which packages would you like to change?
- [ ] `@eslint/compat`
- [ ] `@eslint/config-array`
- [ ] `@eslint/config-helpers`
- [ ] `@eslint/core`
- [ ] `@eslint/mcp`
- [ ] `@eslint/migrate-config`
- [ ] `@eslint/object-schema`
- [x] `@eslint/plugin-kit`
### What problem do you want to solve?
Currently the provided base source code does not provide token-related methods (e.g. `getTokenBefore`).
As a result the source codes of our CSS, JSON and Markdown languages also do not have them.
Further the JSON source code reimplements `getTokenBefore` and `getTokenAfter` manually.
The parsers of all three languages provide access to tokens, so developers should also have access to them.
The tokens are especially helpful for creating robust fixers.
### What do you think is the correct solution?
Exporting a new class `TokenSourceCode` extending the existing `TextSourceCodeBase`.
There are two ways forward for this:
- Implement them separately (similar to the [JSON source code](https://github.com/eslint/json/blob/2202c87c4352c4e581ae6c4dfbb4164ad3e4519e/src/languages/json-source-code.js#L343)). Alone the 3 methods `getTokenBefore`, `getTokenAfter` and `getTokens` would be very helpful.
- Move the [token store](https://github.com/eslint/eslint/blob/main/lib/languages/js/source-code/token-store/index.js) from the JavaScript source code into plugin-kit and then let the JavaScript extend the `TokenSourceCode`
### Participation
- [x] I am willing to submit a pull request for this change.
### AI acknowledgment
- [x] I did not use AI to generate this issue report.
- [ ] (If the above is not checked) I have reviewed the AI-generated content before submitting.
### Additional comments
Is the change big enough to warrant a RFC?
Contributor guide
Assessment
This issue has not been assessed yet.