influxdata / influxdata/kapacitor
Lost default values
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
Hello!
```
# kapacitor version
Kapacitor 1.2.0 (git: master 5408057e5a3493d3b5bd38d5d535ea45b587f8ff)
```
I've follow tick script — https://gist.github.com/juise/15790c2276320554f792a529404b71bb
As you can see, we use default() method like:
```
|default()
.field('threshold_delta_revenue', revenueDeltaThreshold)
```
After that we use it in out alert message like:
```
|alert()
.message('{{ .Level }} revenue difference at {{ .Time }} between \'mnt_compiled_stats\' ({{ index .Fields "old.revenue" | printf "%0.3f" }}) and \'9vdb\' ({{ index .Fields "new.revenue" | printf "%0.3f" }}) is {{ index .Fields "percent_delta_revenue" | printf "%0.3f" }}% threshold {{ index .Fields "threshold_delta_revenue" }}')
.warn(lambda: ("percent_delta_revenue" > revenueDeltaThreshold))
```
But sometimes looks like kapacitor lost default events in various tick scripts:
```
[events_difference:query1] 2017/03/07 11:00:00 D! starting next batch query: SELECT mean(requests) AS request, mean(impressions) AS impression, mean(clicks) AS click, mean(conversions) AS conversion, mean(revenue) AS revenue, mean(cost) AS cost FROM telegraf.autogen.datamart WHERE datamart = 'mnt_compiled_stats' AND time >= '2017-03-07T08:00:00Z' AND time < '2017-03-07T09:00:00Z' GROUP BY time(1h, 0s) fill(0)
[events_difference:query2] 2017/03/07 11:00:00 D! starting next batch query: SELECT mean(request) AS request, mean(impression) AS impression, mean(click) AS click, mean(conversion) AS conversion, sum(revenue) AS revenue, sum(cost) AS cost FROM telegraf.autogen."9vdb" WHERE interval = 'hour' AND type = 'synthetic' AND time >= '2017-03-07T08:00:00Z' AND time < '2017-03-07T09:00:00Z' GROUP BY time(1h, 0s) fill(0)
...
[events_difference:alert22] 2017/03/07 11:00:00 E! error evaluating expression for level WARNING: no field or tag exists for percent_delta_conversion
[events_difference:alert7] 2017/03/07 11:00:00 E! error evaluating expression for level WARNING: no field or tag exists for percent_delta_cost
[events_difference:alert13] 2017/03/07 11:00:00 E! error evaluating expression for level WARNING: no field or tag exists for percent_delta_request
[events_difference:alert7] 2017/03/07 11:00:00 E! error evaluating expression for level WARNING: no field or tag exists for percent_delta_cost
[events_difference:alert13] 2017/03/07 11:00:00 E! error evaluating expression for level WARNING: no field or tag exists for percent_delta_request
[events_difference:alert16] 2017/03/07 11:00:00 E! error evaluating expression for level WARNING: no field or tag exists for percent_delta_impression
...
[events_difference:alert13] 2017/03/07 11:00:00 D! OK alert triggered id:datamart:nil msg:OK request difference at 2017-03-07 08:00:00 +0000 UTC between 'mnt_compiled_stats' (46739007) and '9vdb' (40002185) is %!f(MISSING)% threshold data:&{datamart map[] [time new.click new.conversion new.cost new.impression new.request new.revenue old.click old.conversion old.cost old.impression old.request old.revenue percent_delta_click threshold_delta_click threshold_delta_conversion threshold_delta_cost] [[2017-03-07 08:00:00 +0000 UTC 728546 11227 4650.30268 2.6694919e+07 4.0002185e+07 7424.18391 728548 11234 5053.90939562 2.67363e+07 4.6739007e+07 8847.96388335 0.00027451863158489687 1 1 1]]}
[events_difference:alert10] 2017/03/07 11:00:00 D! WARNING alert triggered id:datamart:nil msg:WARNING revenue difference at 2017-03-07 08:00:00 +0000 UTC between 'mnt_compiled_stats' (8847.964) and '9vdb' (7424.184) is %!f(MISSING)% threshold data:&{datamart map[] [time new.click new.conversion new.cost new.impression new.request new.revenue old.click old.conversion old.cost old.impression old.request old.revenue percent_delta_click threshold_delta_click threshold_delta_conversion threshold_delta_cost] [[2017-03-07 08:00:00 +0000 UTC 728546 11227 4650.30268 2.6694919e+07 4.0002185e+07 7424.18391 728548 11234 5053.90939562 2.67363e+07 4.6739007e+07 8847.96388335 0.00027451863158489687 1 1 1]]}
```
And what we have in out slack alerts:
```
[14:00]
WARNING revenue difference at 2017-03-07 08:00:00 +0000 UTC between 'mnt_compiled_stats' (8847.964) and '9vdb' (7424.184) is %!f(MISSING)% threshold
[14:00]
OK request difference at 2017-03-07 08:00:00 +0000 UTC between 'mnt_compiled_stats' (46739007) and '9vdb' (40002185) is %!f(MISSING)% threshold
[14:00]
OK cost difference at 2017-03-07 08:00:00 +0000 UTC between 'mnt_compiled_stats' (5053.909) and '9vdb' (4650.303) is %!f(MISSING)% threshold 1
```
But sometimes it's work correctly:
```
[12:00]
WARNING revenue difference at 2017-03-07 06:00:00 +0000 UTC between 'mnt_compiled_stats' (9406.509) and '9vdb' (7722.518) is 17.902% threshold 1
[12:00]
WARNING cost difference at 2017-03-07 06:00:00 +0000 UTC between 'mnt_compiled_stats' (5299.467) and '9vdb' (4904.938) is 7.445% threshold 1
```
Contributor guide
Research direction
Start with the TICK script in the linked gist, focusing on the default() and alert() calls, and reproduce the missing-field output shown in the issue. Trace how default values move into alert evaluation and run the script across the affected cases. Done means the referenced fields and thresholds remain available reliably in alert messages.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100