Error Prone Performance Problems.

Hi guys, I'm working on integration of Error Prone into few Lyft Android/Java projects.

During integration I've faced significant build time increase to a point where it became unusable.

I started investigating, benchmarking and profiling different Error Prone setups.

Here is reproducible, open-source benchmarking & profiling setup I've came up with:
https://github.com/artem-zinnatullin/error-prone-performance

It uses RxJava as a test project and integrates Error Prone with different configurations:

- Error Prone as a replacement for JDK's compiler / as javac plugin
- JDK 8 / JDK 9
- Default configuration / Some checks disabled
- etc

Here is short sum of my findings:

- Error Prone adds significant overhead to compilation time
- JDK 8: [up to `3.57`x slower](https://github.com/artem-zinnatullin/error-prone-performance#median-build-time-docker-oracle-jdk-8-180_161-error-prone-replaces-jdks-javac)
- JDK 9: [As compiler plugin up to `2.15`x slower](https://github.com/artem-zinnatullin/error-prone-performance#median-build-time-docker-oracle-jdk-9-9041-error-prone-as-javac-plugin)
- Error Prone adds significant memory overhead on JDK 8 with Parallel GC: [up to `2.58`x more memory used during compilation](https://github.com/artem-zinnatullin/error-prone-performance#max-used-size-memory-docker-oracle-jdk-8-180_161-error-prone-replaces-jdks-javac)

For more info, benchmark & profiling results, instructions & scripts to reproduce it on your end please refer to the [readme of the project](https://github.com/artem-zinnatullin/error-prone-performance)

---

### What seems to cause Error Prone overhead?

According to method call profiling, Error Prone spends significant amount of time on this subset of checks:

- [`PrivateSecurityContractProtoAccess.matchMethodInvocation`](https://github.com/google/error-prone/blob/v2.2.0/core/src/main/java/com/google/errorprone/bugpatterns/PrivateSecurityContractProtoAccess.java) 2.8% - 663 ms (Protobuf)
- [`ImmutableModification.matchMethodInvocation`](https://github.com/google/error-prone/blob/v2.2.0/core/src/main/java/com/google/errorprone/bugpatterns/ImmutableModification.java) 2.4% - 551 ms (Guava)
- [`ShouldHaveEvenArgs.matchMethodInvocation`](https://github.com/google/error-prone/blob/v2.2.0/core/src/main/java/com/google/errorprone/bugpatterns/ShouldHaveEvenArgs.java) 0.8% - 195 ms (Truth)
- [`CollectionIncompatibleType.matchMethodInvocation`](https://github.com/google/error-prone/blob/v2.2.0/core/src/main/java/com/google/errorprone/bugpatterns/collectionincompatibletype/CollectionIncompatibleType.java) 0.6% - 142 ms (JDK)
- [`NamedParameterChecker.matchMethodInvocation`](https://github.com/google/error-prone/blob/v2.2.0/core/src/main/java/com/google/errorprone/bugpatterns/argumentselectiondefects/NamedParameterChecker.java) 0.3% - 62,342 µs (JDK)

While these checks might be relevant to many of the Google's projects, only 2 of them are relevant to the test RxJava project.

Profiling Lyft projects where I tried to intgrate Error Prone gives similar results: lots of time is spent on checks that are not relevant to the codebase in the first place.

#### Potential improvements

##### Laziness

I hope it's possible to optimize these checks by checking compilation classpath and excluding irrelevant checks from analysis. For example, whole category of Guava checks can be excluded if Guava is not found in compilation classpath.

##### Parallelism

I hope it's possible to parallelize AST analysis since Error Prone doesn't modify it by default.

Profiling suggests that compilation step where Error Prone does its work only uses single thread.

---

>Tested on Error Prone 2.2.0.

P.S. I'm not trying to blame Error Prone developers, I'm here to help :) I just want to use it in our setup 😸