getsentry / getsentry/sentry

[Feature Request] Metric alerts to consider events appearance lag

Open
#50,067 5 comments 0 reactions 0 assignees View on GitHub
Product Area: Alerts
Dominant language
Python
Stars
44.8k
Forks
4.9k
Avg merge
1d 5h
Merged PRs (30d)
624

Description

### Problem Statement

I've encountered the following case:
- I want to be notified when the total number of events exceeds a certain Threshold in the interval of 1 hr;
- I created Metric Alert, which is configured for that
- I can clearly see on the Alert page, that the threshold is reached, however alert doesn't fire

All events are coming from mobile clients. And I noticed, that they're coming with the lag. Lets say:
- There's a spike of events at 8 AM
- If I check Alert page at 1 PM - the spike at 8 AM doesn't reach the threshold
- If you check the same page at 10 PM - threshold for 8 AM spike is reached, but alert doesn't fire

I suppose, that I could make the time interval 1 day, but that would not lead to the expected behaviur, cause I'm interested in 1 hr interval checks, not 1 day.

### Solution Brainstorm

I suppose that the problem is that Sentry doesn't let us configure alerts with that lag in mind. So, if I configure it for 1hr interval, it simply checks the last hour. However, if within the same hour it receives retrospective events from mobile clients, and those events lead to threshold reach in the past - Sentry simply doesn't check that.

So, I'd expect from Sentry to allow configure not only alert time interval, but also an interval in the past, which should be avaluated. Lets say:
- Past interval 1 day
- Alert interval 1 hour

When each new event arrives Sentry should check all 1 hour intervals of the previous 1 day for Alerts.

### Product Area

Alerts

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.