influxdata / influxdata/kapacitor

groupBy not grouping by a tag created with eval

Open
#1,765 15 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

- Full details of your operating system (or distribution) e.g. 64-bit Ubuntu 14.04.
- Running InfluxDB/Kapacitor/Chronograf as Docker containers on MacOSX, latest Docker.
- The version of Kapacitor you are running
- Latest, 1.4.
- Whether you installed it using a pre-built package, or built it from source.
- Official Docker container

We are running into an issue with TICKscript and its groupBy behaviour.

We have two sets of measurements, indoor_temperatures and outdoor_temperatures, which we query with a batch.

The queries look as follows:

```js
var out_temp = batch
|query('SELECT mean(temperature) FROM yyyy')
.every(10s)
.period(120d)
.groupBy(time(1h))
.fill(0)

var in_temp = batch
|query('SELECT mean(temperature) FROM xxxx')
.every(10s)
.period(120d)
.groupBy(time(1h))
.fill(0)
```

If we HTTP out both of them, they create the following sets of data:

``` json
{
"series": [
{
"name": "outdoor_temperatures",
"columns": [
"time",
"mean"
],
"values": [
[
"2017-09-20T17:00:00Z",
0
],
[
"2017-09-20T18:00:00Z",
11.5
]
... the rest
]
}
]
}

{
"series": [
{
"name": "indoor_measurements",
"columns": [
"time",
"mean"
],
"values": [
[
"2017-09-20T17:00:00Z",
585.44012944984
],
[
"2017-09-20T18:00:00Z",
592.94890510949
]
... the rest
]
}
]
}
```

Now we do a full join of them, which gives us expected results

```js
out_temp
|join(in_temp)
.as('out_temp_mean', 'in_temp_mean')
.tolerance(5m)
.fill(0)
```

httpOut:

```json
{
"series": [
{
"name": "outdoor_temperatures",
"columns": [
"time",
"in_temp_mean.mean",
"out_temp_mean.mean"
],
"values": [
[
"2017-09-20T17:00:00Z",
586.10175438596,
0
],
[
"2017-09-20T18:00:00Z",
592.94890510949,
11.5
]
... the rest
]
}
]
}
```

Which looks perfect. The issue raises when we want to *round the `out_temp_mean.mean` down and groupBy it*

So we go ahead and extend the script

```
out_temp
|join(in_temp)
.as('out_temp_mean', 'in_temp_mean')
.tolerance(5m)
.fill(0)
|eval(lambda: string(floor("out_temp_mean.mean")))
.as('bucket')
.tags('bucket')
.keep('out_temp_mean.mean', 'in_temp_mean.mean')
```

After which the output STILL looks as it should:

``` json
{
"series": [
{
"name": "outdoor_temperatures",
"columns": [
"time",
"in_temp_mean.mean",
"out_temp_mean.mean",
"bucket"
],
"values": [
[
"2017-09-20T17:00:00Z",
586.99190283401,
0,
"0"
],
[
"2017-09-20T18:00:00Z",
592.94890510949,
11.5,
"11"
]
]
}
]
}
```

Now only thing left is to group the values by the new tag bucket:

```js
out_temp
|join(in_temp)
.as('out_temp_mean', 'in_temp_mean')
.tolerance(5m)
.fill(0)
|eval(lambda: string(floor("out_temp_mean.mean")))
.as('bucket')
.tags('bucket')
.keep('out_temp_mean.mean', 'in_temp_mean.mean')
|groupBy('bucket')
```

After which everything goes awry and we are greeted with `series: null`

```
{
"series": null
}
```

Is this expected behaviour? A bug? Or something else?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the supplied batch, join, eval, tags, and groupBy TICKscript, comparing the output before and after the final groupBy('bucket'). Trace Kapacitor's handling of tags created by eval and grouped afterward. Done means the behavior is explained and, if confirmed as a bug, the grouped result is returned consistently with the reported bucket values.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.