AngularCompiler marks not related files as affected in incremental build

### Which @angular/* package(s) are the source of the bug?

compiler-cli

### Is this a regression?

No

### Description

Any change in main.ts/m1.ts/m2.ts eg changing `console.log(1)` to `console.log(2)` will cause `src/lib/types.ts` to be included in affectedFiles (checked here https://github.com/angular/angular-cli/blob/master/packages/ngtools/webpack/src/ivy/plugin.ts#L569)

Looks like type only file is not included in ignoreForEmit Set nor included in emitted files so IncrementalCompilation can't find it previously emitted files and marks it as affected every time

### Please provide a link to a minimal reproduction of the bug

https://github.com/SebastianPodgajny/ng42543/tree/affected-type-repro

### Please provide the exception or error you saw

_No response_

### Please provide the environment you discovered this bug in

```true
Angular CLI: 12.2.1
Node: 14.16.1
Package Manager: yarn 1.21.1
OS: darwin x64

Angular: 12.2.1
... animations, cli, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, router

Package Version
---------------------------------------------------------
@angular-devkit/architect 0.1202.1
@angular-devkit/build-angular 12.2.1
@angular-devkit/core 12.2.1
@angular-devkit/schematics 12.2.1
@schematics/angular 12.2.1
rxjs 6.6.7
typescript 4.3.5
```

### Anything else?

This is followup to this issue https://github.com/angular/angular/issues/42543 which was partially resolved by JoostK PR https://github.com/angular/angular/pull/42759

Commencez par la reproduction minimale sur la branche affected-type-repro et examinez packages/ngtools/webpack/src/ivy/plugin.ts autour de la ligne 569, puis suivez la manière dont la compilation incrémentale de compiler-cli détermine affectedFiles pour main.ts, m1.ts, m2.ts et src/lib/types.ts. C’est terminé lorsque le fait de remplacer console.log(1) par console.log(2) n’inclut plus types.ts dans affectedFiles.

Rédigé par le modèle d'indexation à partir du texte de l'issue.