microsoft / microsoft/playwright
[Feature]: Fail tests on page errors
- Dominant language
- TypeScript
- Stars
- 96.3k
- Forks
- 6.5k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 180
Description
### 🚀 Feature Request
This is pretty easy on userland, however, I think it is a nice feature for playwright.
Browser errors can be easily swollen, and the test might pass despite them. If you are trying to make high quality code, it is worth making the test fail if the browser throws an error (i.e. on('pageerror') or console.log.msg.type is 'error')
### Example
top level configuration:
`failOnBrowserErrors: true` //option for array of strings that are allowed to pass.
then in a test, you can use a dedicated fixture for specific tests:
test('my example test', (({ allowBrowserErrors }) => {
allowBrowserErrors(); //potentially - specific errors.
// test will fail if the browser errs.
});
### Motivation
We had too many component tests that had some serious errors. Those errors were in the browser - far from sight, far from heart.
As initial step I just "redirected" them to the console, and people start fixing, but also it was very useful for the agent to see the error without the need to launch the browser and access the console, then review all the logs to find errors.
Then came the inevitable, where people did not want to go thru logs, so failing the test was the requested.
Contributor guide
Research direction
Start by tracing the existing pageerror and console message handling, then review how top-level configuration and test fixtures are defined. Determine the supported behavior for failing on browser errors and allowing specific errors, and add coverage for both configuration and per-test exceptions. Done means browser errors reliably fail tests without breaking explicitly allowed cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100