getsentry / getsentry/sentry

Metric Alerts: Prevent failure rate triggers during periods of low volume

Open
#66,988 10 comments 6 reactions 0 assignees View on GitHub
Feature Product Area: Alerts
Dominant language
Python
Stars
44.8k
Forks
4.9k
Avg merge
22h 21m
Merged PRs (30d)
586

Description

### Problem statement

Metric alerts have multiple aggregates which are useful for alerting on total failure rates, such as Failure Rate on the spans dataset or Crash Free User/Session Rate on the releases dataset. These aggregates only support a single threshold, which can result in noisy alerts during periods of low volume where a single failure could trigger the alert.

### Solution brainstorm

We need a way to separately define a minimum number of users, or minimum number of failures, before triggering the alert. One way to achieve this might be to add the ability to define formulas which use multiple aggregate (such as `failure_rate() `and `count(errors)`).

### Original user feedback:

> You can set Failure Rate alerts today by percentage, either by checking the failure rate in a certain period, or checking that the percentage in relation to the last period.
>
> For low volume however, failure rate may spike and send noisy alerts.
We should include an option to define an alert on Failure Rate with the condition that the number of failures (or failures/period) exceeds X.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.