allure-framework / allure-framework/allure-python

Add labels from env for all tests in a run

Open
#721 1 comment 9 reactions 0 assignees View on GitHub
contribute task:new feature theme:core
Dominant language
Python
Stars
815
Forks
260
PR merge metrics
No merged PRs in 30d

Description

#### I'm submitting a ...
- [ ] bug report
- [x] feature request
- [ ] support request => Please do not submit support request here, see note at the top of this template.

#### What is the current behavior?
Currently there is no way to pass environment-related info into all tests in a run without modifying the code base of the tests.

#### What is the expected behavior?
Allure should automatically convert environment variables to labels using the naming convention specified below.

#### What is the motivation / use case for changing the behavior?
It\`s useful to test an application or library across multiple different environments. Such environments may differ in
- a language version
- an operating system
- a platform
- a dependency version
- etc.

The environment-based labels help to distinguish test results produced by the same test in different environments. TestOps can group those results apart with its `env from label` feature.

#### Other information

##### Naming convention
Every environment variable in form `ALLURE_LABEL_=` is translated into the label `` with the value ``.
The label is added for each test in the run.

##### Constraints
To ensure consistency across different allure adapters the following constraints are imposed:
- The environment variables are captured when the test is about to start (see [allure-java](https://github.com/allure-framework/allure-java/blob/e91fb588572ed9397911e988035bc000cd93b354/allure-junit-platform/src/main/java/io/qameta/allure/junitplatform/AllureJunitPlatform.java#L509))

##### Example
Given the following environment variables:
```
ALLURE_LABEL_PYTHON_VERSION=3.11
ALLURE_LABEL_OS=Linux
ALLURE_LABEL_USERNAME=John
ALLURE_LABEL_EMPTY=
```

When the tests are run.

Then the following labels are present in any `*-result.json` file created during the run:
```json
{
...
"labels": [
...
{
"name": "PYTHON_VERSION",
"value": "3.11"
},
{
"name": "OS",
"value": "Linux"
},
{
"name": "USERNAME",
"value": "John"
},
{
"name": "EMPTY",
"value": ""
}
]
}
```

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the environment-label timing reference in allure-junit-platform/src/main/java/io/qameta/allure/junitplatform/AllureJunitPlatform.java and the Python adapter entry points that create result JSON files. Run the existing adapter tests, then verify that each test result receives ALLURE_LABEL_ variables, including empty values, with the expected label names and values.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.