allure-framework / allure-framework/allure3

Allure Jenkins Plugin – Percentage-based Quality Gates not working with Allure 3

Open
#858 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
401
Forks
58
Avg merge
2d 20h
Merged PRs (30d)
34

Description

### What happened?

Jenkins and plugins versions report
When using Allure 3 with the Jenkins Allure Plugin, percentage-based Quality Gates do not behave as expected.
https://plugins.jenkins.io/allure-jenkins-plugin/
The same Quality Gate configuration works correctly with Allure 2. With Allure 3, percentage thresholds are not applied as expected.

Allure 3 correctly reports the test results, but the Jenkins plugin does not appear to evaluate the failure percentage correctly.

What Operating System are you using (both controller, and any agents involved in the problem)?
Java
Gradle
Jenkins Freestyle Job

Reproduction steps
Update allure
Quality gates have the same %, LEAVE_AS_IS 10% if failed 5% for unstable
Run the job
Expected Results
Percentage-based Quality Gates should behave consistently with Allure 2 and correctly evaluate the percentage of failed tests when using Allure 3.

Screenshot 2026-08-04 094025

Actual Results
Percentage-based Quality Gates do not work correctly with Allure 3.

Jenkins Freestyle Job

Failed % is 13% correct in the report

### How can we reproduce it?

Update allure
Quality gates have the same %, LEAVE_AS_IS 10% if failed 5% for unstable
Run the job

### What did you expect?

Expected Results
Percentage-based Quality Gates should behave consistently with Allure 2 and correctly evaluate the percentage of failed tests when using Allure 3.

### Environment

Java
Gradle
Jenkins Freestyle Job
Windows

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start at the linked Jenkins Allure Plugin page and reproduce the percentage-based quality-gate case with Allure 3 in a Windows Jenkins Freestyle Job, using the stated 10% and 5% thresholds. Compare the evaluation with Allure 2 and verify that the reported failed-test percentage produces the expected build result.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
ci-cd, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.