influxdata / influxdata/kapacitor

Alert not working when threshold is reached

Open
#2,257 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

I have created a alert example where if one of 6 devices CPU reach 10% or above. As alert handler I am using a slack channel message that I managed to test it successfully. Here the TICK script:

```var db = 'telegraf'

var rp = 'autogen'

var measurement = 'snmp'

var groupBy = ['hostname']

var whereFilter = lambda: ("hostname" == 'cr1.ams.new.netops.tmcs' OR "hostname" == 'cr1.lon1.netops.tmcs' OR "hostname" == 'cr2.ams.new.netops.tmcs' OR "hostname" == 'cr2.lon1.netops.tmcs' OR "hostname" == 'vfw3-cloudsys-ams1.wr' OR "hostname" == 'vfw3-lon1.netops.tmcs')

var name = 'CPU 80%'

var idVar = name + '-{{.Group}}'

var message = 'CPU usage for {{.Group}} is {{ index .Fields "value" }} at {{.Time}}'

var idTag = 'alertID'

var levelTag = 'level'

var messageField = 'message'

var durationField = 'duration'

var outputDB = 'chronograf'

var outputRP = 'autogen'

var outputMeasurement = 'alerts'

var triggerType = 'threshold'

var crit = 10

var data = stream
|from()
.database(db)
.retentionPolicy(rp)
.measurement(measurement)
.groupBy(groupBy)
.where(whereFilter)
|eval(lambda: "CPUTotal1minRev")
.as('value')

var trigger = data
|alert()
.crit(lambda: "value" >= crit)
.message(message)
.id(idVar)
.idTag(idTag)
.levelTag(levelTag)
.messageField(messageField)
.durationField(durationField)
.slack()
.channel('#netengeu-alerts')

trigger
|eval(lambda: float("value"))
.as('value')
.keep()
|influxDBOut()
.create()
.database(outputDB)
.retentionPolicy(outputRP)
.measurement(outputMeasurement)
.tag('alertName', name)
.tag('triggerType', triggerType)

trigger
|httpOut('output')

Contributor guide

Open the contributing guide

Research direction

Start with the supplied TICKscript and its stream, threshold, alert, and Slack handler configuration. Reproduce the six-device CPU case and verify whether reaching the stated threshold creates an alert and delivers the configured Slack message; the issue is complete when the expected alert behavior is explained or restored.

Written by the indexing model from the issue text.

Assessment

Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.