GoogleCloudPlatform / GoogleCloudPlatform/cloud-opensource-java

Make LinkageChecker faster

Open
#489 1 comment 0 reactions 1 assignee Claimed by @suztomo View on GitHub
enhancement good first issue help wanted p3 Tech Debt
Dominant language
Java
Stars
163
Forks
80
PR merge metrics
No merged PRs in 30d

Description

It seems trying to fetch Maven artifacts from remote repository unnecessarily.

It would take 40000 seconds to fill 167 x 167 table for pair-wise comparison.

```
Running io.grpc:grpc-auth:jar:1.18.0 com.google.cloud:google-cloud-monitoring:jar:1.63.0 (923/27889). ETA: 408741 seconds
Running io.grpc:grpc-auth:jar:1.18.0 com.google.api.grpc:proto-google-cloud-monitoring-v3:jar:1.45.0 (924/27889). ETA: 408741 seconds
Running io.grpc:grpc-auth:jar:1.18.0 com.google.api.grpc:grpc-google-cloud-monitoring-v3:jar:1.45.0 (925/27889). ETA: 408769 seconds
Running io.grpc:grpc-auth:jar:1.18.0 com.google.cloud:google-cloud-os-login:jar:0.81.0-alpha (926/27889). ETA: 408796 seconds
```

There was a bug where each cell of the pair-wise comparison fetching all artifacts in the BOM. Fixed. Now it looks like this:

```
Running com.google.guava:guava:jar:26.0-android com.google.cloud:google-cloud-errorreporting:jar:0.81.0-beta (69/27889). ETA: 40857 seconds
Running com.google.guava:guava:jar:26.0-android com.google.api.grpc:proto-google-cloud-error-reporting-v1beta1:jar:0.46.0 (70/27889). ETA: 41164 seconds
Running com.google.guava:guava:jar:26.0-android com.google.api.grpc:grpc-google-cloud-error-reporting-v1beta1:jar:0.46.0 (71/27889). ETA: 40885 seconds
Running com.google.guava:guava:jar:26.0-android com.google.cloud:google-cloud-firestore:jar:0.81.0-beta (72/27889). ETA: 40717 seconds
```

# DashboardMain

```
Wed Mar 06 16:17:34 EST 2019: reading bom
Wed Mar 06 16:17:38 EST 2019: resolving BOM dependencies
Wed Mar 06 16:17:41 EST 2019: creating linkage checker
Wed Mar 06 16:17:50 EST 2019: finding linkage errors
Wed Mar 06 16:17:57 EST 2019: generating HTML
Wed Mar 06 16:18:02 EST 2019: done.
```

![image](https://user-images.githubusercontent.com/28604/53914401-8a535800-402b-11e9-9640-95b9e1e09468.png)

Even with `session.setOffline(true)`, it remains the same. So the slowness is not from network IO.

```
Wed Mar 06 16:39:52 EST 2019: reading bom
Wed Mar 06 16:39:52 EST 2019: loading artifact info
Wed Mar 06 16:39:56 EST 2019: resolving BOM dependencies
Wed Mar 06 16:39:59 EST 2019: creating linkage checker
Wed Mar 06 16:40:08 EST 2019: finding linkage errors
Wed Mar 06 16:40:15 EST 2019: generating HTML
Wed Mar 06 16:40:20 EST 2019: done.
```

Pairwise comparison table for 167x167 (skipping same version) took 9 hours.

```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 09:29 h
[INFO] Finished at: 2019-03-08T05:41:16-05:00
[INFO] Final Memory: 89M/15503M
[INFO] ------------------------------------------------------------------------

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.