checkstyle / checkstyle/contribution

patch-diff-report-tool: add multi-thread support to CheckstyleReportsParser

Open
#344 5 comments 1 reaction 0 assignees View on GitHub
Dominant language
Java
Stars
57
Forks
168
Avg merge
1d 11h
Merged PRs (30d)
15

Description

Execution:
````
java -jar /home/ricky/opensource/contribution/patch-diff-report-tool/target/patch-diff-report-tool-0.1-SNAPSHOT-jar-with-dependencies.jar --baseReport /home/ricky/regression_reports/savemaster/openjdk10/results.xml --patchReport /home/ricky/regression_reports/savepull/openjdk10/results.xml --output /home/ricky/regression_reports/diff/openjdk10 --baseConfig /var/www/html/reports/32107f44546e135a34db874c19e7cfe4 --patchConfig /var/www/html/reports/32107f44546e135a34db874c19e7cfe4 --refFiles /home/ricky/regression_repositories/openjdk10
patch-diff-report-tool execution started.
XML parsing is started.
````
Config:
````










````
Both report XML files are 169 and 171 megs.

So far I have been waiting 45 minutes for the utility to finish. Based on the displays it has only been working on `CheckstyleReportsParser.parse` this entire time.
I have 4 cores (8 threads) which are sitting idle while I am waiting for this 1 thread to finish basically all the work.

I believe we could multi-thread the following section:
https://github.com/checkstyle/contribution/blob/f4249ac0af8e19a875f450dc7a8fffc7df5405a3/patch-diff-report-tool/src/main/java/com/github/checkstyle/parser/CheckstyleReportsParser.java#L131-L134

Maybe process each file separately with its own thread and memory and then combine the 2 after it is all loaded into memory.

Edit: It finally finished after a bit more than 1 hour.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with patch-diff-report-tool/src/main/java/com/github/checkstyle/parser/CheckstyleReportsParser.java, especially the parse section at lines 131-134. Run the reported command with the two large report XML files and observe the current single-threaded behavior. Done means the parser processes the reports with multi-thread support and still completes the diff successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.