web-platform-tests / web-platform-tests/interop

Kooky Interop idea: Gamify bug reporting

Open
#983 0 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

meta
Dominant language
JavaScript
Stars
524
Forks
35
Avg merge
1d 4h
Merged PRs (30d)
10

Description

This is probably way out of scope for the short-term, but could be a fun direction to explore at some point in the future.

Hypothesis: Many browser bugs go unreported, because there is little incentive to spend the time reducing them and reporting them. I know I'm certainly guilty of this — I report < 10% of the browser bugs I encounter, simply because I cannot justify the additional time.

As a result, a lot of interop issues are actually unknown unknowns. It is not unheard of to see browser features that appear to have good interop on paper (WPT coverage), but not in reality, because their tests are not sufficiently thorough.

However, most web developers struggle to identify browser bugs, reduce them, and report them correctly, so simply encouraging them to report more bugs would result in a lot of noise.

Idea: What if we gamify the process? E.g. start a competition where devs get points for good bug reports, and lose more points for bogus ones to prevent noise. Show a leaderboard throughout and award $$$ to the top N winners at the end.

E.g. something like (strawman):

  • Verified browser bug with testcase: +10 pts (could even have length criteria for the testcase, to avoid "testcases" that are just "I pasted my entire website into CodePen")
  • Verified browser bug without testcase: +1 pts (or even 0)
  • Add testcase to existing bug without one: +5 pts
  • Merged WPT test to test for the bug: +10 pts
  • Duplicate of existing bug: -15 points
  • Invalid bug: -20 points

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named in the proposal. Start by reviewing the Interop issue and current bug-reporting and WPT workflows; a viable contribution would require an agreed competition design, validation rules, and an identified project location.

Written by the indexing model from the issue text.

Assessment

Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.