allure-framework / allure-framework/allure-python

Support python subtests and show failures of subtests in the report

Open
#374 2 comments 7 reactions 0 assignees View on GitHub
task:new feature theme:pytest
Dominant language
Python
Stars
814
Forks
260
PR merge metrics
No merged PRs in 30d

Description

#### I'm submitting a ...
- [x] feature request

#### What is the current behavior?
I'm using soft asserts in my tests. I've implemented those using the subTest python lib.
Example:
```
def __soft_assert_equal(self, a, b, msg):
with self.subTest():
self.assertEqual(a, b, msg)
```
This way if a soft assert fails the tests scripts will go on and execute the next asserts.
I'm running the tests with pytest and using this plugin to show the soft assert failures in the pytest output https://github.com/pytest-dev/pytest-subtests.

However the Allure reports do not show the failures from the subTest and show the test as passed.
Could this be implemented, so Allure reports will show if a assert from a subTest has failed and fail the test in that case? Thanks

#### Please tell us about your environment:

- Allure version: 2.11.0
- Test framework: Python 3.7.3, pytest-4.4.1
- Allure adaptor: allure-pytest-2.6.2

Contributor guide

Open the contributing guide

Research direction

Reproduce the example using Python subTest, pytest, pytest-subtests, and allure-pytest 2.6.2, then inspect the generated Allure report. The work is done when a failed subTest appears in the report and causes the overall test to fail, while later soft assertions still run.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.