dequelabs / dequelabs/axe-core

Flakey tests

Open
#3,561 0 comments 0 reactions 0 assignees View on GitHub
tests
Dominant language
JavaScript
Stars
7.5k
Forks
933
Avg merge
2d 13h
Merged PRs (30d)
29

Description

Recently our tests have been super flaky and timeout a lot. Going to start keeping track of the failing tests to see if I can figure out what's going on.

- [x] IE11 external axe.utils.validLangs - https://app.circleci.com/pipelines/github/dequelabs/axe-core/3825/workflows/b29a4944-bda0-40a8-8984-adc893a08c36/jobs/44782
- [x] Firefox axe.run iframes - https://app.circleci.com/pipelines/github/dequelabs/axe-core/3825/workflows/b29a4944-bda0-40a8-8984-adc893a08c36/jobs/44780
- [ ] Chrome http://localhost:9876/test/integration/full/preload/preload.html - https://app.circleci.com/pipelines/github/dequelabs/axe-core/3824/workflows/c005122b-fbf1-4269-b923-74a7f3683bd6/jobs/44778
- [ ] IE11 color-contrast text-shadow test - https://app.circleci.com/pipelines/github/dequelabs/axe-core/3833/workflows/054e4c88-3120-4b11-aa61-706c788d0731/jobs/44870
- [x] Chrome http://localhost:9876/test/integration/full/isolated-env/isolated-env.html -https://app.circleci.com/pipelines/github/dequelabs/axe-core/3855/workflows/8c57f2be-1832-4a49-b963-dc0f15416242/jobs/45110

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the linked CircleCI jobs for the unchecked Chrome preload test and IE11 color-contrast text-shadow test, then compare them with the already checked failures. Use the reported integration paths and browser names to locate the relevant tests. Done means identifying and resolving the remaining flaky or timing-out cases, with the failures no longer recurring.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.