Support per volume case sensitivity

🔎 Search Terms

"tsserver", "cache invalidation", "new file", "case sensitive", "IntelliSense", "inferred project", "mac os", "watch", "tsserver restart"

🕗 Version & Regression Information

This bug is reproducible between TypeScript 5.1.6 and 5.3.0-nightly (commit b5557271a51704e0469dd86974335a4775a68ffd).

I can test versions prior to 5.1.6 if necessary.

⏯ Playground Link

This bug is specific to a local editing environment and file system quirks. A playground would not be able reproduce this.

🪜 Steps to Reproduce

A few developers on my team have noticed that IntelliSense operations intermittently do not work in newly created files in VS Code. IntelliSense works in the new file after restarting TSServer.

Breakpoint debugging TSServer, I believe there's a file system caching bug related to case sensitivity. The bug can be reproduced consistently on macOS.

1. Open a directory in VS Code with a tsconfig.json file on a case-sensitive volume.
2. Ensure the TypeScript version is set to "Use VS Code's Version". The problem only reproduces when tsserver.js is on a case-insensitive volume, which is usually true for /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/node_modules/typescript/lib/tsserver.js.
3. Create a new file with at least one capital letter in its name. (e.g. Foo.tsx).
4. Observe that IntelliSense operations performed within the new Foo.tsx file do not have information on other files of the same tsconfig.json program.

🙁 Actual behavior

I've created a small repo to help reproduce the bug. There isn't anything special about the repo. It's more where the repo is cloned to and the file system case sensitivity that matters. https://github.com/gluxon/tsserver-new-file-intellisense-bug

Here's a video of the bug.

1. The newly created bar.ts file is able to see IntelliSense information from the rest of the project.
2. The newly created Baz.ts file is not.

https://github.com/microsoft/TypeScript/assets/906558/c43850c1-7b99-40a3-a68a-b2ae8d3d5388

Looking over the TSServer logs for the video above, we can see that bar.ts is matched with the right project. The Baz.ts file is instead mapped to /dev/null/inferredProject2*.

Info 743  [15:52:29.973] Open files: 
Info 743  [15:52:29.973] 	FileName: /Volumes/Code/tsserver-new-file-intellisense-bug/foo.ts ProjectRootPath: /Volumes/Code/tsserver-new-file-intellisense-bug
Info 743  [15:52:29.973] 		Projects: /Volumes/Code/tsserver-new-file-intellisense-bug/tsconfig.json
Info 743  [15:52:29.973] 	FileName: /Volumes/Code/tsserver-new-file-intellisense-bug/bar.ts ProjectRootPath: /Volumes/Code/tsserver-new-file-intellisense-bug
Info 743  [15:52:29.973] 		Projects: /Volumes/Code/tsserver-new-file-intellisense-bug/tsconfig.json
Info 743  [15:52:29.973] 	FileName: /Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts ProjectRootPath: /Volumes/Code/tsserver-new-file-intellisense-bug
Info 743  [15:52:29.973] 		Projects: /dev/null/inferredProject2*

🙂 Expected behavior

TSServer always associates new files with the tsconfig.json file that includes it instead of an inferred project.

---

🔬 Self-Debugging

We noticed that this problem does not reproduce when tsserver.js is loaded from the same case-sensitive file system. It only happens when tsserver.js is loaded from a case-insensitive file system. The main way of changing where tsserver.js was loaded from was through typescript.tsdk in .vscode/settings.json.

With that toggle, we were able to reliably control whether or not the problem reproduces and compare TSServer's internal state for the successful and failing cases. Skip to the bottom for the conclusion.

Program Loading and Caching

Starting at a high-level, we've confirmed in a debugger a few important pieces of information:

1. The TypeScript Program object's rootFilesMap records what files are part of a tsconfig.json. The new file is not added to the expected program's rootFilesMap in the failing case, but is added in the successful case. It makes sense why TSServer associates the newly created file to an inferred project given this.
2. The rootFiles and rootFilesMap fields are computed through getFileNamesFromConfigSpecs(...). This function does not return the newly created file in the failing case (but does in the successful case).
3. The getFileNamesFromConfigSpecs function calls host.readDirectory(...). This readDirectory call does not return the newly created file in the failing case, but does in the successful case.

The host object in getFileNamesFromConfigSpecs is an instance of the CachedDirectoryStructureHost

Debugging CachedDirectoryStructureHost.

The CachedDirectoryStructureHost object is kept up to date through addOrDeleteFileOrDirectory in src/compiler/watchUtilities.ts. We believe the fundamental difference between the failing and successful cases is the evaluation of this line

https://github.com/microsoft/TypeScript/blob/913f65c28da5d4b79180d1a65d1f596f56ae012a/src/compiler/watchUtilities.ts#L315

Failing Case

• The fileOrDirectoryPath const evaluates to /volumes/code/tsserver-new-file-intellisense-bug/baz.ts
• fsQueryResult.fileExists subsequently becomes false.

Note that the entire fileOrDirectoryPath string has been been shifted to be lowercase.

Successful Case

• The fileOrDirectoryPath const evaluates to /Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts
• fsQueryResult.fileExists subsequently becomes true.

In this scenario, the casing of the new file is preserved.

Why is fsQueryResult.fileExists important?

The value of fsQueryResult.fileExists is passed to updateFilesOfFileSystemEntry.

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/compiler/watchUtilities.ts#L324

Once updateFilesOfFileSystemEntry runs, this changes whether or not the CachedDirectoryStructureHost sees the newly created file. I'm able to verify this with a breakpoint on this line, and evaluating the results of readDirectory in the debug console.

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/compiler/watchUtilities.ts#L326

In the successful case

❯ this.readDirectory("/Volumes/Code/tsserver-new-file-intellisense-bug").includes("/Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts")
false

// ...updateFilesOfFileSystemEntry(...)

❯ this.readDirectory("/Volumes/Code/tsserver-new-file-intellisense-bug").includes("/Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts")
true

In the failing case, this always evaluates to false, even after the updateFilesOfFileSystemEntry call.

❯ this.readDirectory("/Volumes/Code/tsserver-new-file-intellisense-bug").includes("/Volumes/Code/tsserver-new-file-intellisense-bug/Baz.ts")
false

Ultimately, I think the problem is before the call to updateFilesOfFileSystemEntry problem though. I would guess updateFilesOfFileSystemEntry should be passed a value of fsQueryResult.fileExists that's true regardless of file system case sensitivity.

How does TypeScript decide when to lowercase?

The fileOrDirectoryPath may or may not be lowercased through:

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/server/editorServices.ts#L1505

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/server/editorServices.ts#L1066-L1068

The this.toCanonicalFileName function performs lower-casing based on this.host.useCaseSensitiveFileNames.

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/compiler/core.ts#L2604

https://github.com/microsoft/TypeScript/blob/a0e0104656415b6c6d18c6ba1100ae42157a4f09/src/server/editorServices.ts#L1028

There's likely an assumption to unwind related to this.host.useCaseSensitiveFileNames. I'm not sure if this is the TypeScript team's preferred fix, but I suspect we need to compute a different toCanonicalFileName function per-project or per-watched folder since the case sensitivity of an opened project folder is independent to where tsserver.js loads from.

Conclusion

The CachedDirectoryStructureHost has a bug where it does not update its cachedReadDirectoryResult internal variable if (1) newly created files have a capital letter and (2) tsserver.js is spawned from a file system volume that's case-insensitive.

Riproduci lo scenario di distinzione tra maiuscole e minuscole di macOS usando il repository collegato, quindi traccia gli aggiornamenti dei file attraverso src/compiler/watchUtilities.ts e getFileNamesFromConfigSpecs in src/compiler/commandLineParser.ts. Confronta i percorsi di canonicalizzazione in src/server/editorServices.ts e src/compiler/core.ts; il lavoro è completato quando i file appena creati con maiuscole e minuscole miste vengono inclusi nel relativo progetto tsconfig invece che in un progetto dedotto, indipendentemente da dove viene caricato tsserver.js.