pytest-dev / pytest-dev/pytest

separate reporting -r and output capture to allow -rA to NOT report output for passing tests

Open
#13,959 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

plugin: capture topic: config type: proposal
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

What's the problem this feature will solve?

The -r flag controls both summary report and output capture reporting.
Really, it'd be nice to have those separate.

For large tests suites, -ra is often used to report the reason for non-passing tests per test in the summary.
For small test suites, -rA is common, as it lists all tests in the summary, including passing.

However, it also reports captured output.
I know there is --show-capture=no. But that turns off showing captured output for failures also.

Describe the solution you'd like

I'd like a way to separate -r functionality into:

  • which outcomes should show summary reports
  • which outcomes should show captured output

However, I know -r is legacy and the default behavior can't change.

But maybe some other way to turn off capture for everything.
These are verbose, but maybe --capture-on that takes chars that matches the -r chars.

So -rA --capture-on=a would report on all, including passing, but capture on all except passing.

The default behavior could stay with "whatever -r" is set to.

Alternative Solutions

Different flag or option name is ok.

Additional context

We had a similar issue with tracebacks, and added --xfail-tb to deal with that.
But output capture is still an issue, and a surprising one, as the documentation for -rA doesn't mention that it turns on passing output capture.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing pytest's -r, -rA, -ra, --show-capture, and --xfail-tb option behavior. Define how summary reporting and captured-output reporting should be selected independently while preserving the current default behavior. Done means passing-test summaries can be shown without passing captured output, while failure output remains available.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.