influxdata / influxdata/kapacitor
BATCH - Eval broken with join
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
kapacitor version : `Kapacitor 1.2.0 (git: master 5408057e5a3493d3b5bd38d5d535ea45b587f8ff)`
OS: `Ubuntu 16.04.1 LTS`
Kernel: `4.4.0-53-generic x86_64`
Hi Team,
When i'm using several batch with a join including eval, eval dont works.
Values
- newvaluepending
- newvalueok
are not present because eval not ran.
Maybe i'm doing something wrong but look like a bug around join using eval.
Thanks for your feedback. All informations are bellow.
See bellow my tick:
```
var data_pending_5min = batch
|query('''select last(value) as value from "metrics_business"."autogen"."reg" where metric='last5min' and result='Pending' ''')
.groupBy('channel','result')
.every(1m)
.period(5m)
.align()
|default()
.field('value', 0)
|log()
.prefix('[DEBUG]-1')
.level('DEBUG')
var data_ok_5min = batch
|query('''select last(value) as value from "metrics_business"."autogen"."reg" where metric='last5min' and result='OK' ''')
.groupBy('channel','result')
.every(1m)
.period(5m)
.align()
|default()
.field('value', 0)
|log()
.prefix('[DEBUG]-2')
.level('DEBUG')
data_ok_5min
|join(data_pending_5min)
.as('data_pending_5min', 'data_ok_5min')
|eval(lambda: "data_pending_5min.value",
lambda: "data_ok_5min.value")
.as('newvaluepending',
'newvalueok')
.keep()
|log()
.prefix('[DEBUG]-3')
.level('DEBUG')
```
Find bellow the kapacitor show:
```
var data_pending_5min = batch
|query('''select last(value) as value from "metrics_business"."autogen"."reg" where metric='last5min' and result='Pending' ''')
.groupBy('channel', 'result')
.every(1m)
.period(5m)
.align()
|default()
.field('value', 0)
|log()
.prefix('[DEBUG]-1')
.level('DEBUG')
var data_ok_5min = batch
|query('''select last(value) as value from "metrics_business"."autogen"."reg" where metric='last5min' and result='OK' ''')
.groupBy('channel', 'result')
.every(1m)
.period(5m)
.align()
|default()
.field('value', 0)
|log()
.prefix('[DEBUG]-2')
.level('DEBUG')
data_ok_5min
|join(data_pending_5min)
.as('data_pending_5min', 'data_ok_5min')
|eval(
lambda: "data_pending_5min.value",
lambda: "data_ok_5min.value"
)
.as(
'newvaluepending',
'newvalueok'
)
.keep()
|log()
.prefix('[DEBUG]-3')
.level('DEBUG')
DOT:
digraph alerting_reg_batch {
graph [throughput="0.00 batches/s"];
query4 [avg_exec_time_ns="0s" batches_queried="2" points_queried="2" query_errors="0" ];
query4 -> default5 [processed="2"];
default5 [avg_exec_time_ns="0s" fields_defaulted="0" tags_defaulted="0" ];
default5 -> log6 [processed="2"];
log6 [avg_exec_time_ns="0s" ];
log6 -> join8 [processed="2"];
query1 [avg_exec_time_ns="0s" batches_queried="2" points_queried="2" query_errors="0" ];
query1 -> default2 [processed="2"];
default2 [avg_exec_time_ns="0s" fields_defaulted="0" tags_defaulted="0" ];
default2 -> log3 [processed="2"];
log3 [avg_exec_time_ns="0s" ];
log3 -> join8 [processed="2"];
join8 [avg_exec_time_ns="0s" ];
join8 -> eval9 [processed="0"];
eval9 [avg_exec_time_ns="0s" eval_errors="0" ];
eval9 -> log10 [processed="0"];
log10 [avg_exec_time_ns="0s" ];
}
```
Log from kapacitor :
```
[alerting_reg_batch:query4] 2017/04/06 16:15:00 D! starting next batch query: SELECT last(value) AS value FROM metrics_business.autogen.reg WHERE metric = 'last5min' AND result = 'OK' AND time >= '2017-04-06T16:10:00Z' AND time < '2017-04-06T16:15:00Z' GROUP BY channel, result
[alerting_reg_batch:query1] 2017/04/06 16:15:00 D! starting next batch query: SELECT last(value) AS value FROM metrics_business.autogen.reg WHERE metric = 'last5min' AND result = 'Pending' AND time >= '2017-04-06T16:10:00Z' AND time < '2017-04-06T16:15:00Z' GROUP BY channel, result
[alerting_reg_batch:log3] 2017/04/06 16:15:00 D! [DEBUG]-1 {"name":"reg","tmax":"2017-04-06T16:15:00Z","group":"channel=channel1,result=Pending","tags":{"channel":"channel1","result":"Pending"},"points":[{"time":"2017-04-06T16:14:01.526306187Z","fields":{"value":16},"tags":{"channel":"channel1","result":"Pending"}}]}
[alerting_reg_batch:log6] 2017/04/06 16:15:00 D! [DEBUG]-2 {"name":"reg","tmax":"2017-04-06T16:15:00Z","group":"channel=channel1,result=OK","tags":{"channel":"channel1","result":"OK"},"points":[{"time":"2017-04-06T16:12:01.732661132Z","fields":{"value":2},"tags":{"channel":"channel1","result":"OK"}}]}
[alerting_reg_batch:log3] 2017/04/06 16:15:00 D! [DEBUG]-1 {"name":"reg","tmax":"2017-04-06T16:15:00Z","group":"channel=channel2,result=Pending","tags":{"channel":"channel2","result":"Pending"},"points":[{"time":"2017-04-06T16:14:01.526306187Z","fields":{"value":8},"tags":{"channel":"channel2","result":"Pending"}}]}
[alerting_reg_batch:log6] 2017/04/06 16:15:00 D! [DEBUG]-2 {"name":"reg","tmax":"2017-04-06T16:15:00Z","group":"channel=channel2,result=OK","tags":{"channel":"channel2","result":"OK"},"points":[{"time":"2017-04-06T16:14:01.526306187Z","fields":{"value":5},"tags":{"channel":"channel2","result":"OK"}}]}
```
Contributor guide
Research direction
Start with the supplied batch tick, focusing on the join and eval nodes, then inspect the DOT output showing join8 processed zero batches. Reproduce the pipeline with the provided queries and logs; done means the joined batches reach eval and the newvaluepending and newvalueok fields appear in the final log output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- data, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100