Alert - Alert fires even though indicators show that it's below threshold
- Dominant language
- Python
- Stars
- 44.8k
- Forks
- 4.9k
- Avg merge
- 22h 21m
- Merged PRs (30d)
- 586
Description
### Environment
SaaS ([https://sentry.io/]())
### Steps to Reproduce
[Customer link provided in internal case]()
But it's a metric alert on P90 span duration for Production data with a specific query
Thresholds are static and action step is to slack.
### Expected Result
Alert should have triggered on higher p90 duration according to the setup of the alert
### Actual Result
both Explore and the Alert dashboard it was triggering on show a lower number so we can't validate if that threshold has been hit which feels like a bug
This happend to both alert 74 as 76
It did mention the alert was modified but I can't check what has been modified and the customer told that they only changed the action step.
### Product Area
Alerts
### Link
[https://www.notion.so/sentry/2738b10e4b5d80c8bce0e865d4d7d464]()
### DSN
*No response*
### Version
*No response*
Contributor guide
Research direction
Start with the Alerts product area and review the linked customer case, then compare the metric alert configuration with the values shown in Explore and the Alert dashboard. Confirm the behavior for alerts 74 and 76 and document which threshold and P90 duration value were evaluated; done means the alert's trigger agrees with the displayed metric or the discrepancy is reproducible with a clear cause.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100