web-platform-tests / web-platform-tests/interop
Kooky Interop idea: Gamify bug reporting
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 524
- Forks
- 35
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 10
Description
This is probably way out of scope for the short-term, but could be a fun direction to explore at some point in the future.
Hypothesis: Many browser bugs go unreported, because there is little incentive to spend the time reducing them and reporting them. I know I'm certainly guilty of this — I report < 10% of the browser bugs I encounter, simply because I cannot justify the additional time.
As a result, a lot of interop issues are actually unknown unknowns. It is not unheard of to see browser features that appear to have good interop on paper (WPT coverage), but not in reality, because their tests are not sufficiently thorough.
However, most web developers struggle to identify browser bugs, reduce them, and report them correctly, so simply encouraging them to report more bugs would result in a lot of noise.
Idea: What if we gamify the process? E.g. start a competition where devs get points for good bug reports, and lose more points for bogus ones to prevent noise. Show a leaderboard throughout and award $$$ to the top N winners at the end.
E.g. something like (strawman):
- Verified browser bug with testcase: +10 pts (could even have length criteria for the testcase, to avoid "testcases" that are just "I pasted my entire website into CodePen")
- Verified browser bug without testcase: +1 pts (or even 0)
- Add testcase to existing bug without one: +5 pts
- Merged WPT test to test for the bug: +10 pts
- Duplicate of existing bug: -15 points
- Invalid bug: -20 points
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named in the proposal. Start by reviewing the Interop issue and current bug-reporting and WPT workflows; a viable contribution would require an agreed competition design, validation rules, and an identified project location.
Written by the indexing model from the issue text.
Assessment
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100