microsoft / microsoft/TypeScript

Performance degrades the more wildcards that are contained in TSConfig include or exclude patterns

Open
#61,103 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Domain: Performance Experimentation Needed Possible Improvement
Dominant language
Go
Stars
111k
Forks
14.3k
Avg merge
2d 4h
Merged PRs (30d)
132

Description

### 🔎 Search Terms

wildcard pattern include exclude tsconfig regex regexp regular expression glob performance slow node

### 🕗 Version & Regression Information

- This is the behavior in every version I tried

### ⏯ Playground Link

_No response_

### 💻 Code

Create a TS project with any empty TS file and the following TSConfig.
```json
{
"compilerOptions": {
"noCheck": true,
"noEmit": true,
"noResolve": true,
},
"exclude": [
"alpha/beta/gamma/delta/epsilon/zeta/eta/theta/iota/kappa/lambda/**/*.ts",
"alpha/beta/gamma/delta/epsilon/zeta/eta/theta/iota/kappa/lambda/**/*.ts",
"alpha/beta/gamma/delta/epsilon/zeta/eta/theta/iota/kappa/lambda/**/*.ts",
...repeat a couple hundred or more times
]
}
```
I've created a repo to more easily reproduce the behavior: https://github.com/MichaelMitchell-at/typescript_slow_wildcards_repro

### 🙁 Actual behavior

`tsc` takes several seconds to run on the project, even though there is no work to do and there are no files to compare the `exclude` patterns against. In fact, running `tsc` with `node --jitless` is much faster than without.

### 🙂 Expected behavior

`tsc` should complete quickly.

### Additional information about the issue

I've determined through profiling that the issue is that a giant `RegExp` combining all the patterns gets created which is very slow to evaluate.
https://github.com/microsoft/TypeScript/blob/739d729ecce60771c23723aad932ab35a34df82d/src/compiler/utilities.ts#L9556-L9566

From my testing, it's much faster to build a separate `RegExp` for each pattern and test them one by one for large number of patterns and for a small number of patterns there is no perceivable difference in speed. I've created a PR implementing this change which greatly improves the performance: https://github.com/microsoft/TypeScript/pull/61104

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/compiler/utilities.ts at lines 9556-9566, then run the linked typescript_slow_wildcards_repro project with the repeated tsconfig exclude patterns. Compare the compiler runtime for large and small pattern sets, and consider the existing PR 61104 when verifying the expected performance improvement.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
compilers, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.