Use the original baseline for % based alerts
- Dominant language
- Python
- Stars
- 44.8k
- Forks
- 4.9k
- Avg merge
- 22h 21m
- Merged PRs (30d)
- 586
Description
### Environment
SaaS ([https://sentry.io/]())
### Steps to Reproduce
Today, %-based metric alerts use the prior interval as the baseline, which means that at any moment, it'll auto-resolve even if the regression is still live, if there's no change from the prior interval.
For example, if I've set up a metric alert for 15-min interval, it'll auto-resolve 15 minutes after firing if there's no change from the prior interval. The more accurate baseline is the original interval that triggered the alert.
### Expected Result
The baseline for the metric alert is based on the original data that triggered the alert.
### Actual Result
Recommended fix:
* Use % comparison to trigger the alert
* Use the % comparison to convert to a number-based threshold
* For example - let's say my trigger is 20% change for same time in a week. If on Monday, 3/1 at 9am, I see that my alert triggers b/c count goes from 200 to 220 week over week, on 3/8, if the count is still at 220, I don't auto-resolve....instead, we we'd check against the original value of 200
Dan's notes:
* "resolution_threadhold is already a AlertRule level property. so we add override_resolve_threshold or something to the Incident and when we fill in a value, then we use that instead of the alert rule's threshold "
Places in code to change:
[https://github.com/getsentry/sentry/blob/e1e99b1ce77904a3970c2f180a72ba10724edf8a/src/sentry/incidents/subscription_processor.py#L147-L178]()
[https://github.com/getsentry/sentry/blob/f93d3584763a8d24e038b6885b0c3ca45e59ea63/src/sentry/incidents/logic.py#L124-L133]()
### Product Area
Alerts
### Link
*No response*
### DSN
*No response*
### Version
*No response*
Contributor guide
Research direction
Start by reading the alert handling in src/sentry/incidents/subscription_processor.py around lines 147-178 and the related logic in src/sentry/incidents/logic.py around lines 124-133. Trace how percentage comparisons become thresholds and how resolution is evaluated. Done means percentage-based alerts retain the original triggering baseline and do not auto-resolve while the regression remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100