perf(usage): stripComments makes import extraction ~11x slower
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 715
- Forks
- 145
- Avg merge
- 21h 39m
- Merged PRs (30d)
- 66
Description
Note: this is an in-house item already being handled by the maintainer - not open for contribution. Filed for tracking only.
Measured while reviewing #1114, which adds a stripComments pass to scanAllImports so JSDoc and commented-out imports stop counting as usage. The pass is correct and worth keeping. It is also about 11x slower than what it replaces.
Measured
src/ plus tests/ in this repo, 271 files, 1.9 MB, averaged over 5 passes:
read only (I/O) : 3.2 ms/pass
old extract : 1.2 ms/pass
new extract : 13.0 ms/pass
So the scanning phase goes from roughly 4.4 ms to 16.2 ms, with the extraction step itself about 11x. At the 5,000-file cap that is roughly 240 ms against 22 ms.
Cause
stripComments builds its result with per-character out += ch over the whole file, materialising a second copy of every source file. The pre-filter in scanAllImports (content.includes("import") || "require" || "export") admits essentially every source file, so almost nothing is skipped.
Options
Accumulate chunks in an array and join("") at the end, or compute comment ranges once and filter regex matches by index rather than building a stripped copy at all. The second avoids the allocation entirely and is probably the better shape, since the caller only needs match positions.
Priority
Not urgent. Absolute numbers are small and correctness matters more than the milliseconds here. Filed because #837 (CLI performance audit) is open in P1 and this is a known, measured, cheap-to-fix regression that should be folded into that work rather than rediscovered later.
Should land after #1114 merges, since it modifies code that PR introduces.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the stripComments pass and its caller, scanAllImports, in the changes introduced by #1114. Reproduce the five-pass measurements on the 271-file src/ and tests/ set, then verify that commented imports remain excluded while the extraction regression is reduced. This item is already being handled by the maintainer and is intended to fold into the #837 performance audit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nodejs, typescript
- Domain
- cli, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 15/100