influxdata / influxdata/kapacitor
Alerting Delays
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
I'm seeing a delay between when data comes into influx and when Kapacitor alerts on this data. Over time I've seen this delay increase to several days. I can't seem to find anything in the logs to indicate why this may happen. I'm streaming data and alerting via smtp. If I restart Kapacitor I get a flood of the alerts which I should have received often days before. I don't see anything to indicate why in my logs, even at debug level. They appear to indicate that current data is being read.
Contributor guide
Research direction
Start by reproducing the delay with streaming data and SMTP alerting while collecting Kapacitor logs at debug level. Compare the alert timing with data arrival, including behavior before and after a Kapacitor restart; done means the cause of the multi-day delay is identified and alerts are delivered on time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100