Metric Alerts: Prevent failure rate triggers during periods of low volume
- Dominant language
- Python
- Stars
- 44.8k
- Forks
- 4.9k
- Avg merge
- 22h 21m
- Merged PRs (30d)
- 586
Description
### Problem statement
Metric alerts have multiple aggregates which are useful for alerting on total failure rates, such as Failure Rate on the spans dataset or Crash Free User/Session Rate on the releases dataset. These aggregates only support a single threshold, which can result in noisy alerts during periods of low volume where a single failure could trigger the alert.
### Solution brainstorm
We need a way to separately define a minimum number of users, or minimum number of failures, before triggering the alert. One way to achieve this might be to add the ability to define formulas which use multiple aggregate (such as `failure_rate() `and `count(errors)`).
### Original user feedback:
> You can set Failure Rate alerts today by percentage, either by checking the failure rate in a certain period, or checking that the percentage in relation to the last period.
>
> For low volume however, failure rate may spike and send noisy alerts.
We should include an option to define an alert on Failure Rate with the condition that the number of failures (or failures/period) exceeds X.
Contributor guide
Assessment
This issue has not been assessed yet.