AngularCompiler marks not related files as affected in incremental build

### Which @angular/* package(s) are the source of the bug?

compiler-cli

### Is this a regression?

No

### Description

Any change in main.ts/m1.ts/m2.ts eg changing `console.log(1)` to `console.log(2)` will cause `src/lib/types.ts` to be included in affectedFiles (checked here https://github.com/angular/angular-cli/blob/master/packages/ngtools/webpack/src/ivy/plugin.ts#L569)

Looks like type only file is not included in ignoreForEmit Set nor included in emitted files so IncrementalCompilation can't find it previously emitted files and marks it as affected every time

### Please provide a link to a minimal reproduction of the bug

https://github.com/SebastianPodgajny/ng42543/tree/affected-type-repro

### Please provide the exception or error you saw

_No response_

### Please provide the environment you discovered this bug in

```true
Angular CLI: 12.2.1
Node: 14.16.1
Package Manager: yarn 1.21.1
OS: darwin x64

Angular: 12.2.1
... animations, cli, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, router

Package Version
---------------------------------------------------------
@angular-devkit/architect 0.1202.1
@angular-devkit/build-angular 12.2.1
@angular-devkit/core 12.2.1
@angular-devkit/schematics 12.2.1
@schematics/angular 12.2.1
rxjs 6.6.7
typescript 4.3.5
```

### Anything else?

This is followup to this issue https://github.com/angular/angular/issues/42543 which was partially resolved by JoostK PR https://github.com/angular/angular/pull/42759

Beginne mit der minimalen Reproduktion auf dem Branch affected-type-repro und untersuche packages/ngtools/webpack/src/ivy/plugin.ts ungefähr bei Zeile 569. Verfolge anschließend, wie die inkrementelle Kompilierung von compiler-cli affectedFiles für main.ts, m1.ts, m2.ts und src/lib/types.ts bestimmt. Erledigt ist die Aufgabe, wenn das Ändern von console.log(1) in console.log(2) types.ts nicht mehr in affectedFiles aufnimmt.

Vom Indexierungsmodell aus dem Issue-Text verfasst.