microsoft / microsoft/TypeScript
⚡ Performance: Project service doesn't cache all fs.realpath
Đang mở
@sheetalkamat đang làm issue này rồi.
Từ ngày 18/7/2024.
- Ngôn ngữ chính
- Go
- Star
- 111k
- Fork
- 14.4k
- Merge trung bình
- 1 ngày 19 giờ
- Pull request đã merge (30 ngày)
- 117
Mô tả
Acknowledgement
- I acknowledge that issues using this template may be closed without further explanation at the maintainer's discretion.
Comment
Acknowledgement
- I acknowledge that issues using this template may be closed without further explanation at the maintainer's discretion.
Comment
🔎 Search Terms
project service fs realpath realpathSync
🙁 Actual behavior
When using typescript-eslint's parserOptions.projectService, type checking APIs switch from the traditional manual TypeScript ts.Program approach to the editor-style ts.ProjectService. We're observing excess calls to the realpath function on some paths in node_modules/ - up to 3-4 for some paths (!).
🙂 Expected behavior
There should be no uncached realpath calls, I'd think?
As a draft, I added a basic caching Map to realpath and ran a before & after comparison with hyperfine (--runs 50). The results showed a ~0.5-2.5% improvement in lint time:
| Variant | Measurement | User Time |
|---|---|---|
| Baseline | 3.153 s ± 0.039 s | 4.403 s |
| Caching | 3.073 s ± 0.048 s | 4.377 s |
diff patch to switch to the Caching variant...
diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
index 4baad59..e53476d 100644
--- a/node_modules/typescript/lib/typescript.js
+++ b/node_modules/typescript/lib/typescript.js
@@ -13,6 +13,8 @@ See the Apache Version 2.0 License for specific language governing permissions
and limitations under the License.
***************************************************************************** */
+var realpathCache = new Map();
+
var ts = {}; ((module) => {
"use strict";
var __defProp = Object.defineProperty;
@@ -8798,6 +8800,15 @@ var sys = (() => {
return path.length < 260 ? _fs.realpathSync.native(path) : _fs.realpathSync(path);
}
function realpath(path) {
+ const cached = realpathCache.get(path);
+ if (cached) {
+ return cached;
+ }
+ const result = realpathWorker(path);
+ realpathCache.set(path, result);
+ return result;
+ }
+ function realpathWorker(path) {
try {
return fsRealpath(path);
} catch {
Additional information about the issue
On the typescript-eslint side:
- https://github.com/typescript-eslint/performance is the performance repository showing how to create these profiles.
- https://github.com/typescript-eslint/performance/blob/73e32fdd3138c479d9e275305aa18e62e88d57be/README.md#comparison-project-service-uncached-file-system-path-reads has the measurements for this specific investigation.
- https://github.com/typescript-eslint/typescript-eslint/pull/6754 is the PR that first added usage of the project service in typescript-eslint.
These are the top 10 most common paths called by realpath...
4 /Users/josh/repos/performance/node_modules/@types/node/index.d.ts
4 /Users/josh/repos/performance/node_modules/prettier
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example0
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example1
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example10
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example10/nested1
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example10/nested10
3 /Users/josh/repos/performance/cases/files-1024-layout-even-singlerun-true-types-service/src/example10/nested10/nested1
Here's an example call stack from the most common one...
Error
at Object.realpath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:8807:33)
at host.compilerHost.realpath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:127075:128)
at realPath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44380:35)
at getOriginalAndResolvedFileName (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:43301:28)
at resolveTypeReferenceDirective (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:43389:74)
at Object.resolve (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:123707:42)
at resolveNamesWithLocalCache (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:129425:29)
at Object.resolveTypeReferenceDirectiveReferences (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:129493:12)
at ConfiguredProject2.resolveTypeReferenceDirectiveReferences (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:182460:33)
at resolveTypeReferenceDirectiveNamesWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124519:20)
at resolveTypeReferenceDirectiveNamesReusingOldState (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124672:14)
at processTypeReferenceDirectives (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125992:156)
at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125881:9)
at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
at processImportedModules (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126134:11)
at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125886:7)
at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
at processImportedModules (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126134:11)
at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125886:7)
at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
at processImportedModules (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126134:11)
at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125886:7)
at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125686:22
at getSourceFileFromReferenceWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125657:26)
at processSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125684:5)
at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125978:7
at forEach (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:2389:22)
at processReferencedFiles (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125977:5)
at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125880:9)
at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125686:22
at getSourceFileFromReferenceWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125657:26)
at processSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125684:5)
at processTypeReferenceDirectiveWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126015:7)
at processTypeReferenceDirective (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126007:5)
at processTypeReferenceDirectives (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126001:7)
at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125881:9)
at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125737:20)
at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125686:22
at getSourceFileFromReferenceWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125657:26)
at processSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125684:5)
at processTypeReferenceDirectiveWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126015:7)
at processTypeReferenceDirective (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126007:5)
at createProgram (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124224:9)
at synchronizeHostDataWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:148948:15)
at synchronizeHostData (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:148844:7)
at Object.getProgram (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:149020:5)
at ConfiguredProject2.updateGraphWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:183144:41)
at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:183000:32)
at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184302:24)
at updateWithTriggerFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184785:11)
at _ProjectService.reloadConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:186401:5)
at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184298:29)
at updateWithTriggerFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184785:11)
at updateConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184793:9)
at _ProjectService.findCreateOrReloadConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187262:44)
at _ProjectService.tryFindDefaultConfiguredProjectForOpenScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187286:25)
at _ProjectService.tryFindDefaultConfiguredProjectAndLoadAncestorsForOpenScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187332:25)
at _ProjectService.assignProjectToOpenedScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187217:27)
at _ProjectService.openClientFileWithNormalizedPath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187411:48)
at _ProjectService.openClientFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187120:17)
at useProgramFromProjectService (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/useProgramFromProjectService.js:61:28)
at getProgramAndAST (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/parser.js:44:100)
at parseAndGenerateServices (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/parser.js:155:11)
at Object.parseForESLint (/Users/josh/repos/performance/node_modules/@typescript-eslint/parser/dist/parser.js:101:80)
at Object.parse (/Users/josh/repos/performance/node_modules/eslint/lib/languages/js/index.js:186:26)
at parse (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:931:29)
at Linter._verifyWithFlatConfigArrayAndWithoutProcessors (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:1696:33)
at Linter._verifyWithFlatConfigArray (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:2062:21)
at Linter.verify (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:1528:61)
at Linter.verifyAndFix (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:2299:29)
at verifyText (/Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:498:48)
at /Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:939:40
at async Promise.all (index 1)
at async ESLint.lintFiles (/Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:880:25)
at async Object.execute (/Users/josh/repos/performance/node_modules/eslint/lib/cli.js:521:23)
at async main (/Users/josh/repos/performance/node_modules/eslint/bin/eslint.js:153:22)
This looks very similar to #59338. My instinct is that it's the same root cause. Filing separately for visibility into the issue & just in case they're not actually the same.
Hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Đánh giá
Issue này chưa được đánh giá.