influxdata / influxdata/influxdb

[InfluxDB - TSDB engine] Negation operators do not work as expected

Open
#15,528 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Issue description:__
There is an issue on TSDB side with handling incoming `Flux` queries which contains negation operators in its body (`!=` or `!~`). Sometimes, the results of such queries execution are incorrect.

__Environment info:__
* System info: `Linux 4.15.0-65-generic x86_64`
* InfluxDB version: `v1.7.7`

__Config:__
Enable Flux:
```
# ...
[http]
# ...
flux-enabled = true
# ...
```

__Steps to reproduce:__
1. Start InfluxDB and write these points to the launched instance (`one measurement - different tags`):
```
curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE monitoring_main"
curl -i -XPOST 'http://localhost:8086/write?db=monitoring_main' --data-binary 'test_metric,host=tagA value=0.78'
curl -i -XPOST 'http://localhost:8086/write?db=monitoring_main' --data-binary 'test_metric,host=tagB value=0.64'
```
2. Execute this `Flux` query to validate that all series have been written successfully:
```
curl -XPOST -H "Content-Type:application/vnd.flux" -H "Accept:application/csv" -d 'from(bucket:"monitoring_main") |> range(start: -1500d) |> filter(fn: (r) => r._measurement == "test_metric")' http://localhost:8086/api/v2/query
```
The result should contain two rows with data:
```
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,dateTime:RFC3339,double,string,string,string
#group,false,false,true,true,false,false,true,true,true
#default,_result,,,,,,,,
,result,table,_start,_stop,_time,_value,_field,_measurement,host
,,0,2015-09-13T14:22:15.539309668Z,2019-10-22T14:22:15.539309668Z,2019-10-22T14:18:32.257030947Z,0.78,value,test_metric,tagA
,,1,2015-09-13T14:22:15.539309668Z,2019-10-22T14:22:15.539309668Z,2019-10-22T14:18:45.148127572Z,0.64,value,test_metric,tagB
```
3. Execute this `Flux` query:
```
curl -XPOST -H "Content-Type:application/vnd.flux" -H "Accept:application/csv" -d 'from(bucket:"monitoring_main") |> range(start: -1500d) |> filter(fn: (r) => r.host != "tagA")' http://localhost:8086/api/v2/query
```

__Expected behavior:__
The result of the query provided above should look like this output:
```
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,dateTime:RFC3339,double,string,string,string
#group,false,false,true,true,false,false,true,true,true
#default,_result,,,,,,,,
,result,table,_start,_stop,_time,_value,_field,_measurement,host
,,0,2015-09-13T14:28:05.121304894Z,2019-10-22T14:28:05.121304894Z,2019-10-22T14:27:04.20388464Z,0.64,value,test_metric,tagB
```

__Actual behavior:__
However, the result of the query provided above is `empty`.

___Note___:
After changing `filter` function's body from `fn: (r) => r.host != "tagA"` to `fn: (r) => r._measurement == "test_metric" and r.host != "tagA"` I've got the desired result:
```
curl -XPOST -H "Content-Type:application/vnd.flux" -H "Accept:application/csv" -d 'from(bucket:"monitoring_main") |> range(start: -1500d) |> filter(fn: (r) => r._measurement == "test_metric" and r.host != "tagA")' http://localhost:8086/api/v2/query
```
Result:
```
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,dateTime:RFC3339,double,string,string,string
#group,false,false,true,true,false,false,true,true,true
#default,_result,,,,,,,,
,result,table,_start,_stop,_time,_value,_field,_measurement,host
,,0,2015-09-13T14:28:05.121304894Z,2019-10-22T14:28:05.121304894Z,2019-10-22T14:27:04.20388464Z,0.64,value,test_metric,tagB
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the standalone Flux filter query against the two points described in the issue, then compare it with the version that also filters on _measurement. Trace the TSDB query path handling != and !~; done when the standalone query returns only the tagB row and regression coverage verifies the behavior.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.