Automatically detect code language in reading mode for better highlighting
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 39k
- Forks
- 2.1k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 10
Description
Clear and concise description of the problem
In reading mode, mozilla/readability causes the code highlighting effect and language annotation attributes to disappear, but I noticed that shiki still highlights the content. It would be better if the language could be automatically detected before highlighting.
Suggested solution
In the past, I ported guesslang, a library that detects the programming language of source code through deep learning, to run in the browser using TensorFlow.js.
I added the esm format and web worker calls to it and used it in a production environment. You can also refer to this online demo:
https://ray-d-song.github.io/guesslang-js/
Alternative
No response
Additional context
No response
Validations
- Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
- This issue is valid
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Trace how reading mode uses mozilla/readability and how shiki receives code before highlighting. Review the referenced guesslang-js demo and its browser, ESM, and web-worker approach, then define how automatic language detection should restore highlighting and language attributes in reading mode.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- tensorflow, typescript
- Domain
- ai, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100