allure-framework / allure-framework/allure3
Diff Viewer Trigger condition
- Dominant language
- HTML
- Stars
- 401
- Forks
- 58
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 34
Description
### What happened?
Hello,
I am a bit confused as how to trigger the diff viewer in my test results since the documentation makes it seem like it should be automatic https://allurereport.org/docs/assertion-diff/#the-diff-viewer
Do i need to use a specific type of assertion?
Does the assert message need to contain a specific keyword?
Is there something to add to the config file for it to show?
Do we have a list of the specific framework that should trigger it?
I have the issue in both Playwright and Pytest tests where i have a comparison assertion but the allure3 diff viewer isnt showing on my Awesome reports (in the error summary at the top or in the test body), I only get the native Playwright/Pytest difference inside the error message.
Playwright:
Python:
### How can we reproduce it?
This is my current config file:
```js
export default defineConfig({
name: reportName,
output: reportOutput,
historyPath: `${testResults}/history.jsonl`,
appendHistory: true,
historyLimit: 25,
categories: {
rules: [
{
name: "Regressions",
id: "regressions",
matchers: { // this section determines what's included into the category
statuses: ["failed"],
transitions: ["regressed"],
},
// this section determines how the category is presented
groupByMessage: true,
expand: false
},
{
name: "Malfunctions",
id: "malfunctions",
matchers: data => data.status === "broken" || data.transition === "malfunctioned",
statuses: ["broken"],
transitions: ["malfunctioned"],
groupByMessage: true,
expand: false
},
{
name: "Timeout",
id: "timeoutsPlaywright",
matchers: {
message: new RegExp('.*TimeoutError.*'),
labels: { "framework": "playwright" }
}
},
{
name: "Timeout",
id: "timeoutsPython",
matchers: {
message: new RegExp('.*Timeout.*'),
labels: { "framework": "pytest" }
}
},
{
id: "other-failed-tests",
name: "Other failed tests",
matchers: {
statuses: ["failed"],
},
groupBy: [],
groupByMessage: false,
groupEnvironments: false,
expand: false,
}
]
},
defaultLabels: { // Does not override, only adds these values when the label doesnt exist on a test
"Criticity": "unknown",
"severity": "No severity"
},
hideLabels: [ // Hide these labels from the test details
"framework",
"package",
"language",
"titlePath",
"host",
"thread"
],
plugins: { // Plugins ids shouldnt use capital letters or the url to access the report wont work properly
severitysuite: {
import: "@allurereport/plugin-awesome",
options: {
reportName: `${reportName} - Grouped by Bug Severity`,
singleFile: false,
reportLanguage: "en",
publish: true,
groupBy: ["severity", "parentSuite", "suite", "subSuite"],
layout: "split",
stepTreeExpansion: "expand_failed_only",
theme: "dark",
charts: customChartList
},
},
qualityGate: {
rules: [
// Values can be set with the pipeline schedule var
{ // Overall success rate must be at least 90%
successRate: parseFloat(minSuccessRate),
},
{ // No more than 2 failed blocker tests
blockerFailures: parseInt(maxBlockerFailures),
},
{ // No more than 5 failed critical tests
criticalFailures: parseInt(maxCriticalFailures),
},
],
use: [
{
...successRateRule,
message: ({ actual, expected }) =>
`Minimum required success rate: ${(expected * 100).toFixed(0)}%. Current success rate is ${(actual * 100).toFixed(1)}%.`,
},
blockerFailuresRule,
criticalFailuresRule,
],
},
variables: {
"Branch Name": "branch_name",
}
});
```
My assert in python is:
```python
assert api_result == expected_output["data"]
```
and in playwright it is:
```ts
await expect.soft(actualId, `Actual row should be ${expected[i].id}'s`).toBe(expectedId);
```
### What did you expect?
To have the diff viewer when my assertion compares texts or objects
### Environment
- Allure 3 version: 3.17.0
- Node.js version: v22.11.0
- Playwright version: 1.63.0
- Python version: 3.11.0
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.