pytest-dev / pytest-dev/pytest
Tweak `--tb=auto` to be equivalent to `--tb=short`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.5k
- Forks
- 3.4k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 35
Description
There's earlier discussion (and the origin of --tb=auto) over in #364.
I'm not sure that the behaviour of --tb=auto is the best default behaviour that we can provide our users:
- It implies un-proportional importance of the first and last frame in a traceback, since they're the only ones whose functions are printed
- A significant amount of time, the last frame is very unhelpful in a traceback because it's down in 3rd-party or standard-library code.
- As projects get larger, the majority of code (and the majority of test risk) is in the code being orchestrated, not the test itself. So, the first frame (the test itself) becomes less likely to be the cause of the failure than the rest of the code running in the test.
- It can be tricky and a little time-consuming to find the "meat of the traceback", especially if the first/last frames in the traceback are within very large functions (I promise I didn't write them! I'm just the unlucky new maintainer 😛)
If we adopt --tb=short to be the default instead:
- It's more familiar, because it's similar to a regular Python exception traceback (though more concise, which I really appreciate. Great work team!)
- The entire stack is visible at once without any scrolling, which makes it easier to understand the shape of the failure.
Alternative Solutions
One way to improve the current situation without anything too radical would be to expand the last first-party code frame in the stack, leaving all 3rd-party and standard-library frames collapsed by default. However, it can be tricky to identify "first-party vs third-party", especially when sys.path modifications are happening (hello from Firefox's build tools! 👋 😄 ).
Additional context
I have a bug reported downstream with additional rationale, some of it project-specific.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the earlier discussion in #364 and compare the current --tb=auto and --tb=short behavior in pytest. This issue needs an agreed traceback policy before implementation; done would include the chosen behavior and its corresponding project changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100