eslint / eslint/rewrite

Change Request: Provide a TokenSourceCode

Open
#465 3 comments 0 reactions 1 assignee Claimed by @DMartens View on GitHub
enhancement
Dominant language
JavaScript
Stars
348
Forks
51
Avg merge
2d 10h
Merged PRs (30d)
11

Description

### Which packages would you like to change?

- [ ] `@eslint/compat`
- [ ] `@eslint/config-array`
- [ ] `@eslint/config-helpers`
- [ ] `@eslint/core`
- [ ] `@eslint/mcp`
- [ ] `@eslint/migrate-config`
- [ ] `@eslint/object-schema`
- [x] `@eslint/plugin-kit`

### What problem do you want to solve?

Currently the provided base source code does not provide token-related methods (e.g. `getTokenBefore`).
As a result the source codes of our CSS, JSON and Markdown languages also do not have them.
Further the JSON source code reimplements `getTokenBefore` and `getTokenAfter` manually.
The parsers of all three languages provide access to tokens, so developers should also have access to them.
The tokens are especially helpful for creating robust fixers.

### What do you think is the correct solution?

Exporting a new class `TokenSourceCode` extending the existing `TextSourceCodeBase`.
There are two ways forward for this:
- Implement them separately (similar to the [JSON source code](https://github.com/eslint/json/blob/2202c87c4352c4e581ae6c4dfbb4164ad3e4519e/src/languages/json-source-code.js#L343)). Alone the 3 methods `getTokenBefore`, `getTokenAfter` and `getTokens` would be very helpful.
- Move the [token store](https://github.com/eslint/eslint/blob/main/lib/languages/js/source-code/token-store/index.js) from the JavaScript source code into plugin-kit and then let the JavaScript extend the `TokenSourceCode`

### Participation

- [x] I am willing to submit a pull request for this change.

### AI acknowledgment

- [x] I did not use AI to generate this issue report.
- [ ] (If the above is not checked) I have reviewed the AI-generated content before submitting.

### Additional comments

Is the change big enough to warrant a RFC?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.