web-platform-tests / web-platform-tests/rfcs

Scoring Reftests

Open
#167 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
110
Forks
90
PR merge metrics
No merged PRs in 30d

Description

I was drafting up advice for when to split out reftests into multiple files or gather them into a single one, and one concern that came in was “combining reftests affects scores in WPT”, which... is not great. :/ Can we make this concern go away somehow?

I would like to be able to give advice based on arguments like “significantly improves understandability of the tests by factoring out common code on related test variants”, or “makes it easier to understand test results by keeping Level 1 and Level 2 features separate”, or “reduces test running time by an order of magnitude”, rather than bringing WPT test scores into it.

(Similarly, I don't want this concern to start weighing into the choice of whether to use testharness.js or reftests. Reftests are far more comprehensive for layout, so there are good reasons to use them.)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the discussion about reftests, testharness.js, and how combining reftests affects WPT scores. Determine the scope and acceptance criteria for removing score-related concerns while preserving clear guidance on test organization and test type selection.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.