influxdata / influxdata/kapacitor
disable global deadman for some scripts
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
Hello,
I have deadman configured globally:
kapacitor.conf:
```toml
[deadman]
# Configure a deadman's switch
# Globally configure deadman's switches on all tasks.
# NOTE: for this to be of use you must also globally configure at least one alerting method.
##global = false
global = true
# Threshold, if globally configured the alert will be triggered if the throughput in points/interval is <= threshold.
threshold = 0.0
# Interval, if globally configured the frequency at which to check the throughput.
interval = "300s"
message = "...."
```
I have a tick script using count():
```batch
var data = stream
|from()
.measurement('xxxxxx')
.groupBy('xxxxxx')
.where(lambda: "xxxx" !~ /xxxxxx/)
data |default()
.field('xxxx', 'xxxx')
.tag('xxxx', '')
|window()
.period(period)
.every(every)
.align()
|count('xxxx')
.as('my_val')
var alert = data
|alert()
...
```
My problem is that it may occur, and this is normal, that I have no value with the ```where``` criteria for long period. Then, count is null, and deadman sends me an alert, and also logs in log file:
```batch
error evaluating expression for level" service=kapacitor task_master=main task=xxxx node=alert5 err="left reference value \"my_val\" is missing value" level=CRITICAL
```
I would like to disable deadman for this script, and also to avoid the logs.
I have tested a lot of things without success.
Is it possible?
Contributor guide
Research direction
No files, tests, or entry points are named. Start by tracing how the global deadman configuration is applied to tasks and how missing count values reach alert evaluation; done means a task can opt out of the global deadman behavior without producing the reported erroneous log.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100