microsoft / microsoft/vscode-textmate
Textmate engine bug for `\k<>` backreferences
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 678
- Forks
- 135
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 2
Description
Example of working as expected
Input is on the left, output is on the right.
The "end" pattern is referencing the 2nd group created in "begin" (the EOF)

What (intentional) failure looks like (non-issue)
A bad pattern causes this kind of behavior:
(note: yellow is the theme's color for entity.shell, which is the included pattern)

What is broken
\k<2> should be equivlent to \2 and in other places it does behave equivlently
However, instead of failing normally (e.g. all-yellow) it seems to trigger undefined behavior:
(Note: \2 is not a viable workaround when group numbers are ≥10)

Here's the code for the problematic pattern.
This is for VS Code 1.72.2, on Mac M1
{
"begin": "(<<)\\s*+\\s*+((?<!\\w)[a-zA-Z_][a-zA-Z_0-9]*(?!\\w))(?=\\s|;|&|<|\"|')",
"end": "\\2",
"beginCaptures": {
"1": {
"name": "keyword.operator.heredoc.shell"
},
"2": {
"name": "string.delimiter.shell"
}
},
"endCaptures": {},
"name": "string.unquoted.heredoc.no-indent.shell",
"patterns": [
{
"match": ".+",
"name": "entity.shell"
}
]
}
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the supplied begin/end grammar, comparing the \k<2> and \2 backreferences in the TextMate engine. Trace how begin and end backreferences are parsed and handled, including group numbers of 10 or more. Done means invalid backreferences fail normally instead of causing the reported undefined behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100