microsoft / microsoft/vscode-textmate
Subroutines breaking capture tokenizing inside of referenced capture group
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 678
- Forks
- 135
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 2
Description
When trying to call a subroutine on a capture group via \\g<1>.
The call will remove all the previous tokens from capture groups that aren't rechecked in the subroutine.
Create a syntax highlighting extension with this code
{
"$schema": "https://raw.githubusercontent.com/martinring/tmlanguage/master/tmlanguage.json",
"name": "Subroutines Syntax",
"scopeName": "source.redcmd.syntax.subroutines",
"patterns": [
{ "include": "#subroutines" }
],
"repository": {
"subroutines": {
"match": "((a)|(b)|(c)|(d))-\\g<1>",
"captures": {
"2": { "name": "strong variable.other.constant" },
"3": { "name": "strong keyword.control" },
"4": { "name": "strong support.type" },
"5": { "name": "strong constant.character.escape" }
}
}
}
}

Expected outcome is that it will highlight all text in the format [abcd]-[abcd]
a-a
a-b
a-c
a-d
b-a
b-b
b-c
b-d
c-a
c-b
c-c
c-d
d-a
d-b
d-c
d-d
Like so:

But instead all tokens connected to capture groups that don't get rematched against (and fail) in the subroutine call get purged.
(capture groups 2 to 5)

Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the JSON grammar and a-a through d-d examples provided in the report. Start by tracing how the tokenizer handles the \\g<1> subroutine and capture-group tokens; done means the expected capture-group highlighting remains visible for all listed combinations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100