web-platform-tests / web-platform-tests/wpt
Tests that compete for globally shared resources are flaky when run in parallel
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 6.2k
- Forks
- 3.9k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 586
Description
Originally filed as https://bugzilla.mozilla.org/show_bug.cgi?id=1866605
In that example we have two tests that use the clipboard. When run in parallel the tests both writing to and reading from the clipboard have an obvious race condition which causes intermittent failures. Our usual approach of partitioning the state doesn't work here because it's OS-level global state; partitioning would be equivalent to requiring tests to be run single-process-per-OS-instance.
To solve this the obvious abstract requirement is some way to annotate which tests are known to require the same shared state. For example:
<meta name=state content=clipboard>
Functionally it would then be up to the runner to ensure that multiple tests declaring the same state requirements don't run in parallel. This could be achieved by each type of state being associated with a mutex that must be acquired before starting the test (should be OK if the number of tests is small and contention is unlikely) or by scheduling such that those tests are never run in parallel.
Requirements wise a question is whether there's a case where we want vendor-specific global state annotations in expectation metadata files. This could be required if a certain browser has implementation-specific global state (e.g. a case where using the media stack from multiple processes at once causes issues).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test entry point is named. Start by locating the test runner's expectation-metadata handling and parallel scheduling, then review how the clipboard example is represented. Done means reaching an agreed design for shared-state annotations and runner behavior, including vendor-specific metadata requirements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- html
- Domain
- testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100