allure-framework / allure-framework/allure3

Diff Viewer Trigger condition

Open
#963 0 comments 0 reactions 1 assignee Claimed by @formaceft-93 View on GitHub
Dominant language
HTML
Stars
401
Forks
58
Avg merge
2d 20h
Merged PRs (30d)
34

Description

### What happened?

Hello,

I am a bit confused as how to trigger the diff viewer in my test results since the documentation makes it seem like it should be automatic https://allurereport.org/docs/assertion-diff/#the-diff-viewer

Do i need to use a specific type of assertion?
Does the assert message need to contain a specific keyword?
Is there something to add to the config file for it to show?
Do we have a list of the specific framework that should trigger it?

I have the issue in both Playwright and Pytest tests where i have a comparison assertion but the allure3 diff viewer isnt showing on my Awesome reports (in the error summary at the top or in the test body), I only get the native Playwright/Pytest difference inside the error message.

Playwright:
Image

Python:
Image

### How can we reproduce it?

This is my current config file:

```js
export default defineConfig({
name: reportName,
output: reportOutput,
historyPath: `${testResults}/history.jsonl`,
appendHistory: true,
historyLimit: 25,
categories: {
rules: [
{
name: "Regressions",
id: "regressions",
matchers: { // this section determines what's included into the category
statuses: ["failed"],
transitions: ["regressed"],
},
// this section determines how the category is presented
groupByMessage: true,
expand: false
},
{
name: "Malfunctions",
id: "malfunctions",
matchers: data => data.status === "broken" || data.transition === "malfunctioned",
statuses: ["broken"],
transitions: ["malfunctioned"],
groupByMessage: true,
expand: false
},
{
name: "Timeout",
id: "timeoutsPlaywright",
matchers: {
message: new RegExp('.*TimeoutError.*'),
labels: { "framework": "playwright" }
}
},
{
name: "Timeout",
id: "timeoutsPython",
matchers: {
message: new RegExp('.*Timeout.*'),
labels: { "framework": "pytest" }
}
},
{
id: "other-failed-tests",
name: "Other failed tests",
matchers: {
statuses: ["failed"],
},
groupBy: [],
groupByMessage: false,
groupEnvironments: false,
expand: false,
}
]
},
defaultLabels: { // Does not override, only adds these values when the label doesnt exist on a test
"Criticity": "unknown",
"severity": "No severity"
},
hideLabels: [ // Hide these labels from the test details
"framework",
"package",
"language",
"titlePath",
"host",
"thread"
],
plugins: { // Plugins ids shouldnt use capital letters or the url to access the report wont work properly
severitysuite: {
import: "@allurereport/plugin-awesome",
options: {
reportName: `${reportName} - Grouped by Bug Severity`,
singleFile: false,
reportLanguage: "en",
publish: true,
groupBy: ["severity", "parentSuite", "suite", "subSuite"],
layout: "split",
stepTreeExpansion: "expand_failed_only",
theme: "dark",
charts: customChartList
},
},
qualityGate: {
rules: [
// Values can be set with the pipeline schedule var
{ // Overall success rate must be at least 90%
successRate: parseFloat(minSuccessRate),
},
{ // No more than 2 failed blocker tests
blockerFailures: parseInt(maxBlockerFailures),
},
{ // No more than 5 failed critical tests
criticalFailures: parseInt(maxCriticalFailures),
},
],
use: [
{
...successRateRule,
message: ({ actual, expected }) =>
`Minimum required success rate: ${(expected * 100).toFixed(0)}%. Current success rate is ${(actual * 100).toFixed(1)}%.`,
},
blockerFailuresRule,
criticalFailuresRule,
],
},
variables: {
"Branch Name": "branch_name",
}
});
```

My assert in python is:
```python
assert api_result == expected_output["data"]
```

and in playwright it is:
```ts
await expect.soft(actualId, `Actual row should be ${expected[i].id}'s`).toBe(expectedId);
```

### What did you expect?

To have the diff viewer when my assertion compares texts or objects

### Environment

- Allure 3 version: 3.17.0
- Node.js version: v22.11.0
- Playwright version: 1.63.0
- Python version: 3.11.0

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.