PrismJS / PrismJS/prism

[v2] Match tokens via named capturing groups

Open
#3,924 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
13k
Forks
1.4k
Avg merge
15h 36m
Merged PRs (30d)
3

Description

Prism's token syntax is helpful for certain things, but for other use cases a single regex with named capturing groups can be easier.
I propose we just use all named capturing groups as token names (automatically converted to kebab-case)
If that becomes a problem, we can introduce a setting to ignore some of them.

This can be combined with regular tokens, they'd just operate on the unmatched text. The tokens matched in the regex can also be specified again in the literal if we want to specify inside or $language. E.g. imagine something like this for the markdown code blocks:

'code-block': {
	pattern: /^```(?<codeLanguage>[a-z-]+).+(?:\n|\r\n?)(?<code>[\s\S]*)?(?:\n|\r\n?)```$/i,
	'punctuation': /```/,
	'code': {
		$language: groups => groups.codeLanguage,
	}
}

Note that this means inside can also be a regex now.

Here's a more substantial example: https://github.com/PrismJS/prism/commit/586bd55e5661a12f9876cb972761f317a5314305

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the issue's Markdown code-block example and commit 586bd55e5661a12f9876cb972761f317a5314305. Define the behavior for converting all named capture groups to kebab-case token names, combining them with regular tokens, and allowing regex-based inside rules and group references; completion requires these cases to work together.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.