influxdata / influxdata/kapacitor
Receiving 2 email alerts for an event
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
I have setup an alert rule with a check on the cpu usage > 40%
I am receiving 2 emails with different levels (CRITICAL when the threshold is crossed i.e. > 40%) and OK when the usage comes back to normal i.e. < 40%.
What I really want is, receive a single email when the CPU usage crosses 40%. Please help me.
Tick Script generated using chronograf:
var db = 'sample'
var rp = 'default_rp'
var measurement = 'cpu'
var groupBy = []
var whereFilter = lambda: TRUE
var name = 'cpu_test'
var idVar = name
var message = ''
var idTag = 'alertID'
var levelTag = 'level'
var messageField = 'message'
var durationField = 'duration'
var outputDB = 'chronograf'
var outputRP = 'autogen'
var outputMeasurement = 'alerts'
var triggerType = 'threshold'
var crit = 40
var data = stream
|from()
.database(db)
.retentionPolicy(rp)
.measurement(measurement)
.groupBy(groupBy)
.where(whereFilter)
|eval(lambda: "usage_user")
.as('value')
var trigger = data
|alert()
.crit(lambda: "value" > crit)
.stateChangesOnly()
.message(message)
.id(idVar)
.idTag(idTag)
.levelTag(levelTag)
.messageField(messageField)
.durationField(durationField)
.email()
trigger
|eval(lambda: float("value"))
.as('value')
.keep()
|influxDBOut()
.create()
.database(outputDB)
.retentionPolicy(outputRP)
.measurement(outputMeasurement)
.tag('alertName', name)
.tag('triggerType', triggerType)
trigger
|httpOut('output')
I also tried to remove .stateChangesOnly() and modify var whereFilter = lambda: ("usage_user" > 40) but in this case, I just received a single alert when the usage crossed 40%. After sometime it came back to normal. But, the next time when it crossed 40%, there was no alert triggered.
Contributor guide
Research direction
Start by reproducing the alert behavior from the provided Kapacitor TICK script, focusing on the alert() configuration, .crit(), and .stateChangesOnly() calls. Compare the generated CRITICAL and OK events with the later behavior when stateChangesOnly is removed; done means alert transitions work consistently on repeated threshold crossings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100