dequelabs / dequelabs/axe-core
Flakey tests
- Dominant language
- JavaScript
- Stars
- 7.5k
- Forks
- 933
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 29
Description
Recently our tests have been super flaky and timeout a lot. Going to start keeping track of the failing tests to see if I can figure out what's going on.
- [x] IE11 external axe.utils.validLangs - https://app.circleci.com/pipelines/github/dequelabs/axe-core/3825/workflows/b29a4944-bda0-40a8-8984-adc893a08c36/jobs/44782
- [x] Firefox axe.run iframes - https://app.circleci.com/pipelines/github/dequelabs/axe-core/3825/workflows/b29a4944-bda0-40a8-8984-adc893a08c36/jobs/44780
- [ ] Chrome http://localhost:9876/test/integration/full/preload/preload.html - https://app.circleci.com/pipelines/github/dequelabs/axe-core/3824/workflows/c005122b-fbf1-4269-b923-74a7f3683bd6/jobs/44778
- [ ] IE11 color-contrast text-shadow test - https://app.circleci.com/pipelines/github/dequelabs/axe-core/3833/workflows/054e4c88-3120-4b11-aa61-706c788d0731/jobs/44870
- [x] Chrome http://localhost:9876/test/integration/full/isolated-env/isolated-env.html -https://app.circleci.com/pipelines/github/dequelabs/axe-core/3855/workflows/8c57f2be-1832-4a49-b963-dc0f15416242/jobs/45110
Contributor guide
Research direction
Start by reviewing the linked CircleCI jobs for the unchecked Chrome preload test and IE11 color-contrast text-shadow test, then compare them with the already checked failures. Use the reported integration paths and browser names to locate the relevant tests. Done means identifying and resolving the remaining flaky or timing-out cases, with the failures no longer recurring.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100