jenkinsci / jenkinsci/gerrit-trigger-plugin

[JENKINS-61750] testWorkflowStepSetsUnsuccessfulMessage failes 2 out of 10 runs

Open

#901 opened on Mar 31, 2020 0 comments 0 reactions 0 assignees View on GitHub

component:gerrit-trigger-plugin imported-jira-issue priority:minor resolution:unresolved

Dominant language: Java
Stars: 226
Forks: 289
PR merge metrics: No merged PRs in 30d

Description

When running the test suites 10 times about 2 times out of 10 the testWorkflowStepSetsUnsuccessfulMessage fails.



butler@jenkins:~/projects/gerrit-trigger-plugin$ cat dotest.sh      

#mvn clean package -DskipTests      

#581  grep elapsed *.out  | sed "s,\(.*sed: \([^ ]*\).*\),\2 \1,g" | sort -n

for i in `seq 1 10` 

do  

time ./doit.sh 2>&1 | tee log.$i.out

done

butler@jenkins:~/projects/gerrit-trigger-plugin$ cat doit.sh

#mvn clean package -DskipTests      

time mvn clean package      

butler@jenkins:~/projects/gerrit-trigger-plugin$ clear      

butler@jenkins:~/projects/gerrit-trigger-plugin$ cat doit.sh

#mvn clean package -DskipTests      

time mvn clean package      

butler@jenkins:~/projects/gerrit-trigger-plugin$ cat dotest.sh      

for i in `seq 1 10` 

do  

time ./doit.sh 2>&1 | tee log.$i.out

done

butler@jenkins:~/projects/gerrit-trigger-plugin$ grep ERR log.*.out 

log.4.out:[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 123.863 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest

log.4.out:[ERROR] testWorkflowStepSetsUnsuccessfulMessage(com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest)  Time elapsed: 16.302 s  <<< FAILURE!

log.4.out:[ERROR]   Run 1: WorkflowTest.testWorkflowStepSetsUnsuccessfulMessage:184

log.8.out:[ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 183.403 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.ParameterModeJenkinsTest

log.8.out:[ERROR] testNameAndEmailParameterModeDefaultChangeAbandoned(com.sonyericsson.hudson.plugins.gerrit.trigger.spec.ParameterModeJenkinsTest)  Time elapsed: 14.454 s  <<< ERROR!

log.8.out:[ERROR]   Run 1: ParameterModeJenkinsTest.testNameAndEmailParameterModeDefaultChangeAbandoned:166 » NullPointer

log.9.out:[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 120.08 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest

log.9.out:[ERROR] testWorkflowStepSetsUnsuccessfulMessage(com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest)  Time elapsed: 16.366 s  <<< FAILURE!

log.9.out:[ERROR]   Run 1: WorkflowTest.testWorkflowStepSetsUnsuccessfulMessage:184

I also find the tests take quite a long time to run (20 minutes on a decent laptop) here are the top 10 "offenders"

// code placeholder

butler@jenkins:~/projects/gerrit-trigger-plugin$ grep elapsed log.1.out | sed "s,$.*sed: \([^ ]*$.*\),\2 \1,g" | sort -n | tail -n 10 | sort -n -r
811.855 [INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 811.855 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.SpecGerritTriggerHudsonTest
231.665 [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 231.665 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.GerritServerHudsonTest
212.626 [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 212.626 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.playback.GerritMissedEventsFunctionalTest
199.747 [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 199.747 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.LockedDownGerritEventTest
179.986 [INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 179.986 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.ParameterModeJenkinsTest
150.987 [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 150.987 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.DuplicateGerritListenersHudsonTestCase
123.204 [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 123.204 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.project.GerritTriggerProjectHudsonTest
118.891 [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.891 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest
105.287 [INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 105.287 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.actions.manual.ManualTriggerActionPermissionTest
102.878 [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.878 s - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.DuplicateGerritListenersPreloadedProjectHudsonTestCase

---
Originally reported by keesj_exset, imported from: testWorkflowStepSetsUnsuccessfulMessage failes 2 out of 10 runs

assignee: rsandell
status: Open
priority: Minor
component(s): gerrit-trigger-plugin
resolution: Unresolved
votes: 0
watchers: 1
imported: 2025-12-07

Raw content of original issue

When running the test suites 10 times about 2 times out of 10 the testWorkflowStepSetsUnsuccessfulMessage fails.

butler@jenkins:~/projects/gerrit-trigger-plugin$ cat dotest.sh
#mvn clean package -DskipTests
#581 grep elapsed *.out | sed "s,$.*sed: \([^ ]*$.*\),\2 \1,g" | sort -n
for i in `seq 1 10`
do
time ./doit.sh 2>&1 | tee log.$i.out
done
butler@jenkins:~/projects/gerrit-trigger-plugin$ cat doit.sh
#mvn clean package -DskipTests
time mvn clean package
butler@jenkins:~/projects/gerrit-trigger-plugin$ clear
butler@jenkins:~/projects/gerrit-trigger-plugin$ cat doit.sh
#mvn clean package -DskipTests
time mvn clean package
butler@jenkins:~/projects/gerrit-trigger-plugin$ cat dotest.sh
for i in `seq 1 10`
do
time ./doit.sh 2>&1 | tee log.$i.out
done
butler@jenkins:~/projects/gerrit-trigger-plugin$ grep ERR log.*.out
log.4.out:[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 123.863 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest
log.4.out:[ERROR] testWorkflowStepSetsUnsuccessfulMessage(com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest) Time elapsed: 16.302 s <<< FAILURE!
log.4.out:[ERROR] Run 1: WorkflowTest.testWorkflowStepSetsUnsuccessfulMessage:184
log.8.out:[ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 183.403 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.spec.ParameterModeJenkinsTest
log.8.out:[ERROR] testNameAndEmailParameterModeDefaultChangeAbandoned(com.sonyericsson.hudson.plugins.gerrit.trigger.spec.ParameterModeJenkinsTest) Time elapsed: 14.454 s <<< ERROR!
log.8.out:[ERROR] Run 1: ParameterModeJenkinsTest.testNameAndEmailParameterModeDefaultChangeAbandoned:166 » NullPointer
log.9.out:[ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 120.08 s <<< FAILURE! - in com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest
log.9.out:[ERROR] testWorkflowStepSetsUnsuccessfulMessage(com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.WorkflowTest) Time elapsed: 16.366 s <<< FAILURE!
log.9.out:[ERROR] Run 1: WorkflowTest.testWorkflowStepSetsUnsuccessfulMessage:184

I also find the tests take quite a long time to run (20 minutes on a decent laptop) here are the top 10 "offenders"

environment

```
commit c0cec29a93c1e6c071df36b99f494bb09a8b30d4 (origin/master, origin/HEAD, master)

Author: rsandell <email>

Date: Fri Feb 14 13:53:46 2020 +0100

[maven-release-plugin] prepare for next development iteration

```

Contributor guide

Open the contributing guide

Research direction

Start by running the Maven suite repeatedly and inspect WorkflowTest.testWorkflowStepSetsUnsuccessfulMessage, reported at line 184. Compare the intermittent failure logs with the related ParameterModeJenkinsTest error and determine what test behavior is nondeterministic. Done means the named workflow test no longer fails intermittently during repeated suite runs.

Written by the indexing model from the issue text.

Assessment

Tech stack: java
Domain: testing-qa
Issue type: Bug
Difficulty: 3/5
Estimated time: 1-2 days
Activity status: Stale
Clarity: Mostly clear
Newbie friendliness: 35/100

[JENKINS-61750] testWorkflowStepSetsUnsuccessfulMessage failes 2 out of 10 runs

Description

Contributor guide

Research direction

Assessment

Get new issues in your inbox