influxdata / influxdata/kapacitor

Kapacitor "show <task name>" shows no data points consumed, but "stats ingress" and strace do

Open
#1,455 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

**OS:** CentOS Linux release 7.2.1511 (Core)
**Kapacitor and InfluxDB Versions:** 1.0.0

Built and installed from your source (no special/weird make options specified):
https://dl.influxdata.com/kapacitor/releases/kapacitor-1.0.0_linux_amd64.tar.gz

## Issue:

`kapacitor list tasks` shows the task running and enabled:
```
ID Type Status Executing Databases and Retention Policies
core-iperf3-bandwidth batch enabled true ["pipe-apps"."pipe-apps"]
```

`kapacitor stats ingress` shows points received:
```
pipe-apps pipe-apps iperf3 145
```

`kapacitor show core-iperf3-bandwidth` shows 0 batches collected:
```
ID: core-iperf3-bandwidth
Error:
Template:
Type: batch
Status: enabled
Executing: true
Created: 23 Jun 17 17:59 UTC
Modified: 23 Jun 17 19:16 UTC
LastEnabled: 23 Jun 17 19:16 UTC
Databases Retention Policies: ["pipe-apps"."pipe-apps"]
TICKscript:
// core-iper3-bandwidth

// Email parameters
var emailAddress =

var msg_details = '''

Kapacitor





Notification Type:
{{ if eq .Level "INFO" }}Info{{ else }}Alert{{ end }}


iPerf3 Client Instance:
{{ index .Tags "host" }}


AWS Region:
{{ index .Tags "region" }}


Kapacitor Task Name:
{{ .TaskName }}


Check State:
{{ .Level }}


Event Time:
{{ .Time }}

'''

// Parameters
var critBits = 104857600

var window = 60m

var frequency = 15m

var buckets = 30m

var renotify = 15m

// Dataframe
var data = batch
|query('''
SELECT mean(sent_bps)
AS stat
FROM "pipe-apps"."pipe-apps"."iperf3"
WHERE "host" =~ /.*/
''')
.period(window)
.every(frequency)
.groupBy(time(buckets), 'host')

// Thresholds
var alert = data
|alert()
.id('{{ index .Tags "host" }}')
.message('Average network bandwidth for {{ .ID }} is {{ .Level }}: {{ index .Fields "stat" }}/Bits per sec')
.details(msg_details)
.stateChangesOnly(renotify)
.crit(lambda: "stat" < critBits)

// Alert
alert
.email(emailAddress)

DOT:
digraph core-iperf3-bandwidth {
graph [throughput="0.00 batches/s"];

query1 [avg_exec_time_ns="0" batches_queried="9" connect_errors="0" points_queried="26" query_errors="0" ];
query1 -> alert2 [processed="9"];

alert2 [alerts_triggered="0" avg_exec_time_ns="0" crits_triggered="0" infos_triggered="0" oks_triggered="0" warns_triggered="0" ];
}
```

Also, when I strace and filter on `read()` calls, I see the kapacitor process is reading the query results in question from our influxdb instance:

```
[pid 26143] <... read resumed> "POST /write?consistency=&db=&precision=ns&rp= HTTP/1.1\r\nHost: \r\nUser-Agent: InfluxDBClient\r\nContent-Length: 966\r\nContent-Type: \r\nAccept-Encoding: gzip\r\n\r\niperf3,dirty_data=false,host=,instance_id=,receiving_server=,region=us-east-1 received_bps=1039223000,sent_bps=1040483000 1498250287000000000\niperf3,dirty_data=false,host=,instance_id=,receiving_server=,region=us-east-1 received_bps=1055083000,sent_bps=1056186000 1498250287000000000\niperf3,dirty_data=false,host=,instance_id=,receiving_server=,region=us-east-1 received_bps=60238092,sent_bps=61656726 1498250287000000000\niperf3,dirty_data=true,host=,instance_id=,receiving_server=,region=us-east-1 received_bps=0,sent_bps=78082 1498250287000000000\niperf3,dirty_data=true,host=,instance_id=,receiving_server=,region=us-east-1 received_bps=0,sent_bps=78083 1498250287000000000\n", 4096) = 1192
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the discrepancy with `kapacitor stats ingress`, `kapacitor show core-iperf3-bandwidth`, and the provided batch query, then compare the DOT statistics with the InfluxDB response observed through `strace`. Done means identifying why ingested points and query results are not reflected as consumed batches and documenting or testing the corrected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.