GoogleCloudPlatform / GoogleCloudPlatform/cloud-opensource-java

Investigate the slowness in building a class path for spring-cloud-gcp-starter-pubsub

Open
#367 1 comment 0 reactions 1 assignee Claimed by @suztomo View on GitHub
enhancement p3
Dominant language
Java
Stars
163
Forks
80
PR merge metrics
No merged PRs in 30d

Description

To reproduce, here is argument for StaticLinkageChecker:

```
-m
https://repo.spring.io/milestone
-a
org.springframework.cloud:spring-cloud-gcp-starter-pubsub:1.1.0.RC2
```

This takes forever (more than 5 minutes, before I give up)

# Investigation

## BFS loop

When checked with debugger, breadth-first iteration in `DependencyGraphBuilder.levelOrder` has more than 20,000 items in the queue. Something seems wrong.

![image](https://user-images.githubusercontent.com/28604/51705431-a1913580-1fe9-11e9-9c98-25a13f1fe2c5.png)

## Does this happen with Maven central?

Maven central has 1.1.0.RELEASE. It can cause the problem.

Arguments:
```
-a org.springframework.cloud:spring-cloud-gcp-starter-pubsub:1.1.0.RELEASE
```

### Duplicate in BFS loop?

There were duplicate Maven artifacts in the BFS loop. After avoiding the duplicate, the number of visited Maven artifacts is now 3781. Created a class path of 1466 elements.

## Run for a night
I started Tuesday night 5:00pm and it still running at Wednesday 9:00 am. (more than 16 hours)

## JProfiler

`DefaultRepositorySystem.collectDependencies` occupies the most time. This is to fetch dependencies.

![image](https://user-images.githubusercontent.com/28604/51944310-aab74380-23e9-11e9-95e6-cf1679c1a911.png)

The method is called every dependency node in the tree. In the case of spring-cloud-gcp-starter-pubsub,

## Are there duplicate HTTP fetch for same artifact many times?

To check.

It does make many HTTP request while iterating 2 million items (2 million after waiting for 1 hour)

![image](https://user-images.githubusercontent.com/28604/52010894-a439d200-24a4-11e9-86f1-f731c38d6854.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.