pytest-dev / pytest-dev/pytest
running counter of test results at -2
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
Currently, at both -1 and -2 the test cases reporting uses the "sea of dots".
For test suites which are both very large and relatively slow, this representation is not really helpful as counting dozens of dots is difficult, and the only easy to understand progress reporting is at end of line.
The "default" progress reporting yields a more precise view as it checkpoints the percentage every file, but that's at the cost of enormous amounts of space if the suite uses a large number of files (which is a good reason to use the dots).
I would propose an alternative representation, which is a lot terser and easier to track for slow test suites (though not necessarily for very small or fast ones): display a running counter in the same format as the summary stats line (maybe with the number of selected tests added) as the sole tally line e.g.
collected 5 items
1 passed / 5 selected
which on the next test would become
2 passed / 5 selected
and on failure
1 failed / 2 passed / 5 selected
This provides for a much improved "at a glance" summary when running tests interactively.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating pytest's -1 and -2 progress-reporting implementation and the code that formats summary statistics. Compare the existing dot and end-of-line modes while running a large, slow test suite. Done means an interactive running tally reports selected, passed, and failed tests without the sea of dots or per-file percentage output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100