Allow asynchronously skipping tests
- Dominant language
- TypeScript
- Stars
- 45.5k
- Forks
- 6.7k
- Avg merge
- 17h 22m
- Merged PRs (30d)
- 24
Description
## 🚀 Feature Proposal
Other testing frameworks allow tests to asynchronously decide whether they should skip themselves.
For example, in Mocha:
```js
it('tests a remote service', async function() {
if (!(await remoteService.isActive())) {
return this.skip()
}
… test the remote service …
})
```
Currently, however, it is impossible to asynchronously decide whether a test should be skipped in Jest.
See also: discussion here: https://github.com/facebook/jest/issues/7245
## Motivation
Some tests either depend on - or are explicitly testing - remote services which may or may not be available.
Without being able to programatically and asynchronously decide whether tests can be skipped, there are only three options for writing these sorts of tests:
1. Decide that they will either always pass or always fail if the service is unavailable. In either case the result can be misleading (ie, because in many cases "failure" indicates "the service is wrong", not merely "the service is unavailable", and "passing" suggests that everything is okay, which is also not necessarily true).
2. Keep them in a separate suite, one per remote service, which can be run with (for example), `npm run test:service-a`).
3. Use a regular expression (or similar) to include / exclude these tests from a test run.
## Example
A complete, real-world (but anonymized) example from a Mocha-based test suite:
```js
describe('example.com', () => {
beforeAll(async function() {
try {
await fetch('https://example.com/api/whoami')
} catch (e) {
return this.skip(`Skipping tests for example.com: ${e}`)
}
})
it('returns the current user', async () => { … })
it('does the other thing', async () => { … })
})
```
## Pitch
This belongs in Jest core because:
1. It's currently supported by Mocha: https://mochajs.org/ (search for `this.skip`)
2. It's impossible to implement outside of core (see elaboration below)
3. The related discussion on #7245 suggests that it's important to a number of people (see, ex, this comment, which as of this writing has 16 👍 : https://github.com/facebook/jest/issues/7245#issuecomment-432240159)
## FAQ
### Why can't you use an if-statement?
A common suggestion in #7245 is to use an if-statement (or similar) to skip tests:
```js
describe('example.com', () => {
const isActive = remoteService.isActive()
if (isActive) it('returns the current user', async () => { … })
})
```
However, this will not work for asynchronous tests, as tests must be declared synchronously, but the "is a remote service active?" check is necessarily asynchronous.
### Wouldn't it be better if the tests failed/succeeded/retried/did something else?
There are situations when this is true, but (as evidenced by discussion on #7245) there are also situations where "skip tests when a remote service is not available" is a reasonable business decision (ex: https://github.com/facebook/jest/issues/7245#issuecomment-474587219)
Contributor guide
Research direction
No repository files or tests are named; start with the related discussion in issue #7245 and compare Jest's current behavior with Mocha's asynchronous `this.skip`. Done means a test can await a remote-service check and be reported as skipped when the service is unavailable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, typescript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100