influxdata / influxdata/kapacitor

Understanding Kapacitor’s deadman Act

Open
#1,433 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

Hi, I'm using telegraf to supervise some process. So I thought it could be interesting if kapacitor is able to notify me when any process die.

I think this can be done with Kapacitor's deadman switch (if a process die, telegraf's procstat input can't send data to influxdb). But i'm having troubles with this, if I kill the process, Kapacitor will send the alert, but when I restart the process, Kapacitor still sending the alert.

I'm trying something like this:

|deadman(1.0, 10s)
.slack()
.channel('#alerts-staging')

If no point has been arrived in 10s, send the alert ( I think it works like that).

But executing kapacitor show in the task, I can see graph [throughput="0.00 points/s"];
I don't know if this thoughput matters.

I'm using kapacitor 1.3.1-1 on Debian 8.

Thanks.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported behavior with Kapacitor 1.3.1-1, the procstat input, and the deadman task shown in the issue. Check the task output from kapacitor show, including throughput, before and after restarting the supervised process. Done means determining why the alert does not clear and documenting or fixing the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
debian
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.