influxdata / influxdata/kapacitor

question about how to calculate percent throught specific tag

Open
#963 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

I have a measurement like this:

```
time city_id come_from desc count
2016-10-09T10:43:36.5502871Z "1" "4" "11" 1
2016-10-09T10:43:39.833652904Z "1" "4" "16" 1
2016-10-09T10:44:37.864675526Z "1" "4" "2" 1
```

In this measurement,I'd like to calculate each `desc`'s percent in total.
Below is the tick

```
var inner = batch
|query('select sum("count") as count from "hestia"."default"."eleme_order_invalid_description"')
.every(1d)
.period(1d)
.groupBy(time(1d), 'desc')
.align()
.fill(0)

var total = batch
|query('select sum("count") as count from "hestia"."default"."eleme_order_invalid_description"')
.every(1d)
.period(1d)
.groupBy(time(1d))
.align()
.fill(0)
|log()
.level('debug')
.prefix('total')

total
|join(inner)
.tolerance(1d)
.as('total', 'inner')
.fill(0.0)
|log()
.level('debug')
.prefix('after join')
|eval(lambda: "inner.count" / "total.count")
.as('rate')
|log()
.level('debug')
.prefix('after evel')
|influxDBOut()
.database('hestia')
.retentionPolicy('default')
.measurement('downsample_marco_invalid_order_description')
```

Strange thing is the `total` not match the `inner` though the same time .Here is the logs:

```
[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=13","tags":{"desc":"13"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":39,"total.count":0},"tags":{"desc":"13"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=14","tags":{"desc":"14"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":162,"total.count":0},"tags":{"desc":"14"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=16","tags":{"desc":"16"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":278,"total.count":0},"tags":{"desc":"16"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=2","tags":{"desc":"2"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":2,"total.count":0},"tags":{"desc":"2"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=7","tags":{"desc":"7"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":1,"total.count":0},"tags":{"desc":"7"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=8","tags":{"desc":"8"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":3,"total.count":0},"tags":{"desc":"8"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=0","tags":{"desc":"0"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":73,"total.count":0},"tags":{"desc":"0"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=1","tags":{"desc":"1"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":1,"total.count":0},"tags":{"desc":"1"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=6","tags":{"desc":"6"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":1,"total.count":0},"tags":{"desc":"6"}}]}

[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","group":"desc=4","tags":{"desc":"4"},"points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":1,"total.count":0},"tags":{"desc":"4"}}]}

//the total , not match the inners
[downsample_marco_invalid_order_description:log6] 2016/10/10 18:51:20 D! after join {"name":"eleme_order_invalid_description","tmax":"2016-10-09T00:00:00Z","points":[{"time":"2016-10-09T00:00:00Z","fields":{"inner.count":0,"total.count":561},"tags":null}]}

```

But if I just add `come_from` within groupBy in `total` and `inner`,it works.

Is there any way to calculate percent the `desc` tag?

Kapacitor version is `Kapacitor 1.0.2 (git: master 1011dba109bf3d83366c87873ec285c7f9140d34)`

Contributor guide

Open the contributing guide

Research direction

Start with the provided Kapacitor batch/query, groupBy, join, and eval pipeline and compare the logged timestamps and group keys for total and inner. Reproduce the behavior on Kapacitor 1.0.2, then determine whether the join should match these streams; done means documenting or correcting the percent calculation behavior.

Written by the indexing model from the issue text.

Assessment

Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.