influxdata / influxdata/kapacitor

how to configure the deadman function to trigger a warning alert instead of a critical alert in kapacitor

Open
#2,802 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

Goal is to configure the deadman function in Kapacitor to trigger a warning alert instead of a critical alert, when there is no metrics form the measurement , Looks like deadman by default when there is no metrics, it is triggering critical alert. tried to run the script with Level tag and without level tag to trigger only warning when there is no metrics but still no luck. Appreciate some ideas to get this fixed.

var data = stream
|from()
.measurement(measurement)
.where(where_filter)
.groupBy(group_by)
|window()
.period(period)
.every(every)
|default()
.field(field_upper, 0.0)

var nodata = data
|deadman(deadman_threshold, deadman_interval)
.id(id)
.message(message)
.stateChangesOnly()
//.levelTag('level')
//.warn(lambda: "level" == 'CRITICAL')
//.warnReset(lambda: "level" == 'OK'
.warn(lambda: "emitted" <= deadman_threshold)
.warnReset(lambda: "emitted" > deadman_threshold)
.topic(topic) // Comment out .topic to disable alert sending and uncomment .log to write alerts to file
//.log('/tmp/alerts.log')

Contributor guide

Open the contributing guide

Research direction

Start with Kapacitor's deadman function behavior and the TICKscript shown in the issue, focusing on alert levels, levelTag, and the warn/warnReset predicates. Confirm how deadman handles missing metrics and what configuration produces a warning-level alert; the issue is complete when that behavior and configuration are documented clearly.

Written by the indexing model from the issue text.

Assessment

Domain
observability
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.