web-platform-tests / web-platform-tests/rfcs
Scoring Reftests
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 110
- Forks
- 90
- PR merge metrics
- No merged PRs in 30d
Description
I was drafting up advice for when to split out reftests into multiple files or gather them into a single one, and one concern that came in was “combining reftests affects scores in WPT”, which... is not great. :/ Can we make this concern go away somehow?
I would like to be able to give advice based on arguments like “significantly improves understandability of the tests by factoring out common code on related test variants”, or “makes it easier to understand test results by keeping Level 1 and Level 2 features separate”, or “reduces test running time by an order of magnitude”, rather than bringing WPT test scores into it.
(Similarly, I don't want this concern to start weighing into the choice of whether to use testharness.js or reftests. Reftests are far more comprehensive for layout, so there are good reasons to use them.)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the discussion about reftests, testharness.js, and how combining reftests affects WPT scores. Determine the scope and acceptance criteria for removing score-related concerns while preserving clear guidance on test organization and test type selection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100