checkstyle / checkstyle/contribution
patch-diff-report-tool: add multi-thread support to CheckstyleReportsParser
- Dominant language
- Java
- Stars
- 57
- Forks
- 168
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 15
Description
Execution:
````
java -jar /home/ricky/opensource/contribution/patch-diff-report-tool/target/patch-diff-report-tool-0.1-SNAPSHOT-jar-with-dependencies.jar --baseReport /home/ricky/regression_reports/savemaster/openjdk10/results.xml --patchReport /home/ricky/regression_reports/savepull/openjdk10/results.xml --output /home/ricky/regression_reports/diff/openjdk10 --baseConfig /var/www/html/reports/32107f44546e135a34db874c19e7cfe4 --patchConfig /var/www/html/reports/32107f44546e135a34db874c19e7cfe4 --refFiles /home/ricky/regression_repositories/openjdk10
patch-diff-report-tool execution started.
XML parsing is started.
````
Config:
````
````
Both report XML files are 169 and 171 megs.
So far I have been waiting 45 minutes for the utility to finish. Based on the displays it has only been working on `CheckstyleReportsParser.parse` this entire time.
I have 4 cores (8 threads) which are sitting idle while I am waiting for this 1 thread to finish basically all the work.
I believe we could multi-thread the following section:
https://github.com/checkstyle/contribution/blob/f4249ac0af8e19a875f450dc7a8fffc7df5405a3/patch-diff-report-tool/src/main/java/com/github/checkstyle/parser/CheckstyleReportsParser.java#L131-L134
Maybe process each file separately with its own thread and memory and then combine the 2 after it is all loaded into memory.
Edit: It finally finished after a bit more than 1 hour.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with patch-diff-report-tool/src/main/java/com/github/checkstyle/parser/CheckstyleReportsParser.java, especially the parse section at lines 131-134. Run the reported command with the two large report XML files and observe the current single-threaded behavior. Done means the parser processes the reports with multi-thread support and still completes the diff successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100