Improved markup, CSS, JS tokenization
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 13k
- Forks
- 1.4k
- Avg merge
- 15h 36m
- Merged PRs (30d)
- 3
Description
Hi! First, thanks of all for this awesome project! It's affected so many projects around the world. Incredible work!
Motivation
I'm trying to recreate the default VS Code syntax highlighting with Prism.js (via prism-react-renderer) and I'm running into limitations of Prism.js.
Description
I would like to be able to highlight certain tokens that are not granular enough in HTML, CSS and JS.
Would it be something that the Prism.js team is interested in to have more granular tokenization of HTML, CSS and JS? If possible, also with backwards compatibility?
Some first quick examples of things that could be improved across HTML, CSS and JS (compared against VS Code 1.33.1 default dark mode):
HTML: DOCTYPE
HTML: Treating quotes and equals differently than normal punctuation
CSS: URL values
CSS: Arguments in selectors
CSS: Keywords - ex. generic font family names, colors (maybe in CSS-Extra?)
JS: Regex sequences, regex escaping
The code used:
<!DOCTYPE html>
<html lang="en">
<head>
<script>
window.console && console.log('foo');
function initHighlight(block, cls) {
try {
if (cls.search(/\bno\-highlight\b/) != -1)
return process(block, true, 0x0f) + ` class="${cls}"`;
} catch (e) {
/* handle exception */
}
for (var i = 0 / 2; i < classes.length; i++) {
if (checkCondition(classes[i]) === undefined)
console.log('undefined');
}
}
</script>
<title>Prism</title>
<style>
@font-face {
src: url(https://lea.verou.me/logo.otf);
font-family: 'LeaVerou';
}
#features li:nth-child(odd),
footer p {
font: 100% Consolas, 'Andale Mono', monospace;
background-image: linear-gradient(
45deg,
transparent 34%,
white 34%,
white 66%,
transparent 66%
);
text-shadow: 0 1px white;
}
</style>
</head>
</html>
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by reviewing Prism's existing HTML, CSS, and JavaScript tokenization against the supplied examples and VS Code comparison, then establish the required token categories and backward-compatibility expectations before implementation; done requires an agreed scope and corresponding coverage for the requested cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- css, html, javascript
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100