allure-framework / allure-framework/allure3
Allure Jenkins Plugin – Percentage-based Quality Gates not working with Allure 3
- Dominant language
- HTML
- Stars
- 401
- Forks
- 58
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 34
Description
### What happened?
Jenkins and plugins versions report
When using Allure 3 with the Jenkins Allure Plugin, percentage-based Quality Gates do not behave as expected.
https://plugins.jenkins.io/allure-jenkins-plugin/
The same Quality Gate configuration works correctly with Allure 2. With Allure 3, percentage thresholds are not applied as expected.
Allure 3 correctly reports the test results, but the Jenkins plugin does not appear to evaluate the failure percentage correctly.
What Operating System are you using (both controller, and any agents involved in the problem)?
Java
Gradle
Jenkins Freestyle Job
Reproduction steps
Update allure
Quality gates have the same %, LEAVE_AS_IS 10% if failed 5% for unstable
Run the job
Expected Results
Percentage-based Quality Gates should behave consistently with Allure 2 and correctly evaluate the percentage of failed tests when using Allure 3.
Actual Results
Percentage-based Quality Gates do not work correctly with Allure 3.
Jenkins Freestyle Job
Failed % is 13% correct in the report
### How can we reproduce it?
Update allure
Quality gates have the same %, LEAVE_AS_IS 10% if failed 5% for unstable
Run the job
### What did you expect?
Expected Results
Percentage-based Quality Gates should behave consistently with Allure 2 and correctly evaluate the percentage of failed tests when using Allure 3.
### Environment
Java
Gradle
Jenkins Freestyle Job
Windows
### Additional context
_No response_
Contributor guide
Research direction
Start at the linked Jenkins Allure Plugin page and reproduce the percentage-based quality-gate case with Allure 3 in a Windows Jenkins Freestyle Job, using the stated 10% and 5% thresholds. Compare the evaluation with Allure 2 and verify that the reported failed-test percentage produces the expected build result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100