Needinfo triage-owners and/or test authors when a test reaches a certain intermittency threshold
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 68
- Forks
- 92
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 16
Description
Currently sheriffs monitor intermittent bugs. If the rate of failure is too high, they needinfo the triage-owner to investigate. For example:
https://bugzilla.mozilla.org/show_bug.cgi?id=1616418#c34
It would be nice if bugbug could handle this needinfo automatically. This would:
- Reduce sheriff workload
- Reduce friction between sheriffs / developers, as the request would be automated
The bot should make it clear that if no action is taken to fix the test, it will be disabled.
The intermittency data is stored in treeherder. I'm unsure what the API call to retrieve it is offhand. But looks like the /failurecount or /failuresbybug endpoints might be the ones we need.
Currently sheriffs needinfo the triage-owner of the component containing the test, but it might be better if we could try to needinfo whoever modifies the test most (and maybe fallback to triage-owner if that person can't be detected or has needinfos disabled).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing Treeherder's /failurecount and /failuresbybug endpoints and the linked intermittent-failure example to determine the available data. Define the intermittency threshold, recipient selection and fallback behavior, then verify that automation needinfos the appropriate person and clearly warns that the test may be disabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100