mozilla / mozilla/bugbot

Needinfo triage-owners and/or test authors when a test reaches a certain intermittency threshold

Open
#1,126 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area-Intermittents
Dominant language
Python
Stars
68
Forks
92
Avg merge
2d 2h
Merged PRs (30d)
16

Description

Currently sheriffs monitor intermittent bugs. If the rate of failure is too high, they needinfo the triage-owner to investigate. For example:
https://bugzilla.mozilla.org/show_bug.cgi?id=1616418#c34

It would be nice if bugbug could handle this needinfo automatically. This would:

  1. Reduce sheriff workload
  2. Reduce friction between sheriffs / developers, as the request would be automated

The bot should make it clear that if no action is taken to fix the test, it will be disabled.

The intermittency data is stored in treeherder. I'm unsure what the API call to retrieve it is offhand. But looks like the /failurecount or /failuresbybug endpoints might be the ones we need.

Currently sheriffs needinfo the triage-owner of the component containing the test, but it might be better if we could try to needinfo whoever modifies the test most (and maybe fallback to triage-owner if that person can't be detected or has needinfos disabled).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing Treeherder's /failurecount and /failuresbybug endpoints and the linked intermittent-failure example to determine the available data. Define the intermittency threshold, recipient selection and fallback behavior, then verify that automation needinfos the appropriate person and clearly warns that the test may be disabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.