pytest-dev / pytest-dev/pytest

pytest.mark.parametrize: parameter=value pairs in test IDs

Open
#13,055 0 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic: parametrize type: proposal
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

What's the problem this feature will solve?

I have a lot of parametrized tests. By default, only the parameter's values make in into the test IDs. The parameter names don't. Since especially bool and int parameters do not speak for themselves (certain strings might not, too), the test function + test ID are not descriptive/expressive.

Describe the solution you'd like

I think it would be beneficial to allow parameter=value pairs in the test ID (optionally) to get an idea what parameters a test gets passed.

The test IDs usually look like this:

test_something[100-10-True-False-True]

While the True/False values could potentially be turned into some other data type with a proper __str__() method (enums, ..), I think having the option to show key=value instead of only the value would be useful to easily see what parameters a test gets passed, e.g.:

test_something[speed_down=100-speed_up=10-foo=True-bar=False-baz=True]

The "-" separator does not look natural with this approach, maybe it could also be configurable? But I think that part is less important here.

I imagine something like pytest.mark.parametrize(.., id_names=True) could enable such behavior.

Alternative Solutions

For the time being, I'm using a wrapper around pytest.mark.parametrize():

def parametrize(argnames, argvalues, **kwargs):
    argnames_consumable = argnames.split(",") * len(argvalues)

    def idfn(value):
        argname = argnames_consumable.pop(0)
        return f"{argname}={value}"

    if "ids" not in kwargs:
        kwargs["ids"] = idfn

    return pytest.mark.parametrize(argnames, argvalues, **kwargs)                                  

This gives me the parameter=value test IDs as described above (if no ids kwarg is given). While I don't consider this a too ugly hack, a solution for this directly in pytest.mark.parametrize() might also be useful to other users.

Additional context

https://docs.pytest.org/en/latest/reference/reference.html#pytest-mark-parametrize
https://docs.pytest.org/en/latest/reference/reference.html#pytest.Metafunc.parametrize

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pytest.mark.parametrize and pytest.Metafunc.parametrize reference entries linked in the issue, then trace how parameter IDs are currently produced. Done should mean an agreed optional way to display parameter=value pairs in test IDs, with the separator behavior and interaction with custom ids clearly defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.