Compile module to a cache directory. Reuse cached output if inputs the same

Suggestion

It would be great if typescript had support for caching to a cache directory and reusing the cached compiler output if the input is the same. This would happen on a per-module basis

Search Terms

output cache, cache directory

Use Cases

This would be extremely useful for monorepos, especially on CI. It would shorten the average build times a lot

Examples

We built a simple prototype / proof of concept here: https://github.com/hfour/ctsc/

Here are the main advantages to project mode, pasted here:

Project mode:

• requires that all modules compile with declarations. This means that the toplevel application packages cannot easily be included in the project mode and have to be compiled separately.
• requires you to specify the dependant projects, even though they can be read from package.json. This means double book-keeping.
• only reuses files found in the local directory. This means it cannot be used to speed up compilation in CI by having a shared cache directory containing most of the unchanged modules, precompiled.
• does extra work than it was asked to (if dependencies have changed it will recompile them). ctsc will work even after the package is removed from a monorepo into a separate repo (see below). It only makes decisions for a single package, based on all the inputs.

Our method works even with external node modules distributed with their own type definitions, as long as those external modules were compiled with ctsc. A .ctsc.hash will be distributed with them, and will result in a change of the input if the dependency has changed its .d.ts files, making the cached output no longer valid. It could be additionally extended so that if a .ctsc.hash is not found, its automatically calculated from .d.ts files, but thats currently not done for performance reasons

We've seen a reduction in compile time for our monorepo from 48s for a cold cache down to 4s for a fully hot cache, and 7-10s for typical single-module changes. A lot of changes don't propagate a recompile further than one depth level of dependencies. Cold cache compile times went up around 10%, to 52s.

Why should this be built-in with typescript? Because for a given compilation invocation, the typescript compiler knows best which files were consulted as inputs and which were generated as outputs. Its the only code that can reliably keep track of them.

Checklist

My suggestion meets these guidelines:

• This wouldn't be a breaking change in existing TypeScript/JavaScript code
• This wouldn't change the runtime behavior of existing JavaScript code
• This could be implemented without emitting different JS based on the types of the expressions
• This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
• This feature would agree with the rest of TypeScript's Design Goals.

edit: In particular, this non-goal:

Provide an end-to-end build pipeline. Instead, make the system extensible so that external tools can use the compiler for more complex build workflows.

can be debated. The question is, how far should TSC go in providing the necessary facilities to implement this? At minimum it seems that a list of the input files consulted and a report of the output files generated is necessary, but just getting the list of input files already involves basic parsing of all input files, which is slow.

Beginnen Sie mit der Überprüfung des Kompilierungsverhaltens von TypeScript im project-mode und des verlinkten ctsc-Prototyps. Konzentrieren Sie sich dabei darauf, wie Compiler-Eingaben und erzeugte Ausgaben nachverfolgt werden könnten. Das Issue nennt keine Quelldateien oder Tests; als abgeschlossen würde gelten, dass der zwischengespeicherte Output pro Modul nur dann wiederverwendet wird, wenn die Eingaben übereinstimmen, einschließlich der Fälle mit gemeinsamem Cache und der Invalidierung von Abhängigkeiten.