influxdata / influxdata/kapacitor

Alerting Delays

Open
#1,918 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

I'm seeing a delay between when data comes into influx and when Kapacitor alerts on this data. Over time I've seen this delay increase to several days. I can't seem to find anything in the logs to indicate why this may happen. I'm streaming data and alerting via smtp. If I restart Kapacitor I get a flood of the alerts which I should have received often days before. I don't see anything to indicate why in my logs, even at debug level. They appear to indicate that current data is being read.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the delay with streaming data and SMTP alerting while collecting Kapacitor logs at debug level. Compare the alert timing with data arrival, including behavior before and after a Kapacitor restart; done means the cause of the multi-day delay is identified and alerts are delivered on time.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.