influxdata / influxdata/influxdb

Inconsistent group key behavior when specifying parameters in a record/object

Open
#24,748 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

When parameters of a Flux query are specified in a record/object and although the queries are semantically identical, the group key is different resulting in different query results.

__Steps to reproduce:__
1. Import this simple dataset (written in line-protocol) in a `Test` bucket:
```js
weather,location=San\ Francisco temp=51.9,pm=38i 1710000000000000000
weather,location=New\ York temp=18.2,pm=0i 1710000000000000000
weather,location=Hong\ Kong temp=53.6,pm=171i 1710000000000000000
```

2. __[Query A]__ Calculate the mean temperature:
```flux
option now = () => 2024-03-10T12:00:00Z

from(bucket: "Test")
|> range(start: -10d)
|> filter(fn: (r) => r._measurement == "weather" and r._field == "temp")
|> group()
|> mean()
```
![Basic query result](https://github.com/influxdata/influxdb/assets/22419043/a983dadd-e7b9-4f84-b9f4-0899d106d565)

Annotated CSV result

```csv
#group,false,false,true,true,false
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,double
#default,_result,,,,
,result,table,_start,_stop,_value
,,0,2024-02-29T12:00:00Z,2024-03-10T12:00:00Z,41.23333333333333
```

3. __[Query B]__ Perform the same query by specifying the parameters in a record:
```flux
option now = () => 2024-03-10T12:00:00Z

p = {
myField: "temp",
myOperation: mean
}

from(bucket: "Test")
|> range(start: -10d)
|> filter(fn: (r) => r._measurement == "weather" and r._field == p.myField)
|> group()
|> p.myOperation()
```
__`_start` and `_stop` columns are missing__ because they are __not part of the group key__, for some unknown reason.
![Parameterized query result](https://github.com/influxdata/influxdb/assets/22419043/707275f1-09f0-4d60-8891-e554fb36e816)

Annotated CSV result

```csv
#group,false,false,false
#datatype,string,long,double
#default,_result,,
,result,table,_value
,,0,41.23333333333333
```

4. __[Query C]__ In some cases, even though the parameters are in a record, the query result is correct:
```flux
option now = () => 2024-03-10T12:00:00Z

p = {
myBucket: "Test",
myOperation: mean
}

from(bucket: p.myBucket)
|> range(start: -10d)
|> filter(fn: (r) => r._measurement == "weather" and r._field == "temp")
|> group()
|> p.myOperation()
```
![Parameterized query result that works](https://github.com/influxdata/influxdb/assets/22419043/31766dbe-9dba-4803-b411-e67ca3e299fa)

Annotated CSV result

```csv
#group,false,false,true,true,false
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,double
#default,_result,,,,
,result,table,_start,_stop,_value
,,0,2024-02-29T12:00:00Z,2024-03-10T12:00:00Z,41.23333333333333
```

5. __[Query D]__ Another version of the same query where the mean function is declared in an anonymous function.
```flux
option now = () => 2024-03-10T12:00:00Z

p = {
myField: "temp",
myOperation: (tables=<-) => tables |> mean()
}

from(bucket: "Test")
|> range(start: -10d)
|> filter(fn: (r) => r._measurement == "weather" and r._field == p.myField)
|> group()
|> p.myOperation()
```
![Parameterized query result that also works](https://github.com/influxdata/influxdb/assets/22419043/31766dbe-9dba-4803-b411-e67ca3e299fa)

Annotated CSV result

```csv
#group,false,false,true,true,false
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,double
#default,_result,,,,
,result,table,_start,_stop,_value
,,0,2024-02-29T12:00:00Z,2024-03-10T12:00:00Z,41.23333333333333
```

__Expected behaviour:__
The same result for queries no matter how the parameters are provided.

__Actual behaviour:__
The group key is different even though the queries are sementically identical.

__Environment info:__
* InfluxDB OSS v2.7.5 (also tested on v2.7.1) running on Windows 11 23H2 (build 22631.3235).
* Same issue on InfluxDB Cloud (powered by TSM) with Storage Engine Version 2 (167c4c6).

Contributor guide

Open the contributing guide

Research direction

Start by reproducing Query A and Query B against the stated InfluxDB dataset and compare their group keys and annotated CSV output. Trace how record parameters and function values are evaluated in the Flux query path; done means semantically equivalent queries preserve the same _start and _stop group-key columns.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.