jenkinsci / jenkinsci/junit-plugin
[JENKINS-10175] "Recording test results" is super slow, what is it doing?
- Dominant language
- Java
- Stars
- 87
- Forks
- 351
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 2
Description
After a successful build I see
Recording test results
It then takes ages to do whatever its doing, sometime spending another 10 minutes shuffling away.
Here is a threadDump:
java.io.WinNTFileSystem.list(Native Method)
java.io.File.list(File.java:937)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1230)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1184)
org.apache.tools.ant.DirectoryScanner.checkIncludePatterns(DirectoryScanner.java:1003)
org.apache.tools.ant.DirectoryScanner.scan(DirectoryScanner.java:882)
org.apache.tools.ant.types.AbstractFileSet.getDirectoryScanner(AbstractFileSet.java:490)
org.apache.tools.ant.types.AbstractFileSet.getDirectoryScanner(AbstractFileSet.java:451)
hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:105)
hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:87)
hudson.FilePath.act(FilePath.java:758)
hudson.FilePath.act(FilePath.java:740)
hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:83)
hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:123)
hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:135)
hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:664)
hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:639)
hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:617)
hudson.model.Build$RunnerImpl.post2(Build.java:161)
hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:586)
hudson.model.Run.run(Run.java:1400)
hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
hudson.model.ResourceController.execute(ResourceController.java:88)
hudson.model.Executor.run(Executor.java:146)
Any chance it could be speeded up? A build which takes 23 minutes ends up taking 33 just because we'd like it to show the junit results, but this seems excessive. We have about 2000 unit tests. Is this a lot?
Thanks.
---
Originally reported by fancellu, imported from: "Recording test results" is super slow, what is it doing?
Raw content of original issue
After a successful build I see
Recording test results
It then takes ages to do whatever its doing, sometime spending another 10 minutes shuffling away.
Here is a threadDump:
java.io.WinNTFileSystem.list(Native Method)
java.io.File.list(File.java:937)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1230)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1259)
org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1184)
org.apache.tools.ant.DirectoryScanner.checkIncludePatterns(DirectoryScanner.java:1003)
org.apache.tools.ant.DirectoryScanner.scan(DirectoryScanner.java:882)
org.apache.tools.ant.types.AbstractFileSet.getDirectoryScanner(AbstractFileSet.java:490)
org.apache.tools.ant.types.AbstractFileSet.getDirectoryScanner(AbstractFileSet.java:451)
hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:105)
hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:87)
hudson.FilePath.act(FilePath.java:758)
hudson.FilePath.act(FilePath.java:740)
hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:83)
hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:123)
hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:135)
hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:664)
hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:639)
hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:617)
hudson.model.Build$RunnerImpl.post2(Build.java:161)
hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:586)
hudson.model.Run.run(Run.java:1400)
hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
hudson.model.ResourceController.execute(ResourceController.java:88)
hudson.model.Executor.run(Executor.java:146)Any chance it could be speeded up? A build which takes 23 minutes ends up taking 33 just because we'd like it to show the junit results, but this seems excessive. We have about 2000 unit tests. Is this a lot?
Thanks.
Contributor guide
Research direction
Start with the JUnitParser.java stack entry and the JUnitResultArchiver.java call path, then inspect the Apache Ant DirectoryScanner activity shown in the thread dump. Reproduce the delay with a build containing many test results and identify a measurable improvement in result-recording time without changing the published results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- performance, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100