pytest-dev / pytest-dev/pytest
pytest.mark.parametrize: parameter=value pairs in test IDs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
What's the problem this feature will solve?
I have a lot of parametrized tests. By default, only the parameter's values make in into the test IDs. The parameter names don't. Since especially bool and int parameters do not speak for themselves (certain strings might not, too), the test function + test ID are not descriptive/expressive.
Describe the solution you'd like
I think it would be beneficial to allow parameter=value pairs in the test ID (optionally) to get an idea what parameters a test gets passed.
The test IDs usually look like this:
test_something[100-10-True-False-True]
While the True/False values could potentially be turned into some other data type with a proper __str__() method (enums, ..), I think having the option to show key=value instead of only the value would be useful to easily see what parameters a test gets passed, e.g.:
test_something[speed_down=100-speed_up=10-foo=True-bar=False-baz=True]
The "-" separator does not look natural with this approach, maybe it could also be configurable? But I think that part is less important here.
I imagine something like pytest.mark.parametrize(.., id_names=True) could enable such behavior.
Alternative Solutions
For the time being, I'm using a wrapper around pytest.mark.parametrize():
def parametrize(argnames, argvalues, **kwargs):
argnames_consumable = argnames.split(",") * len(argvalues)
def idfn(value):
argname = argnames_consumable.pop(0)
return f"{argname}={value}"
if "ids" not in kwargs:
kwargs["ids"] = idfn
return pytest.mark.parametrize(argnames, argvalues, **kwargs)
This gives me the parameter=value test IDs as described above (if no ids kwarg is given). While I don't consider this a too ugly hack, a solution for this directly in pytest.mark.parametrize() might also be useful to other users.
Additional context
https://docs.pytest.org/en/latest/reference/reference.html#pytest-mark-parametrize
https://docs.pytest.org/en/latest/reference/reference.html#pytest.Metafunc.parametrize
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the pytest.mark.parametrize and pytest.Metafunc.parametrize reference entries linked in the issue, then trace how parameter IDs are currently produced. Done should mean an agreed optional way to display parameter=value pairs in test IDs, with the separator behavior and interaction with custom ids clearly defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100