influxdata / influxdata/influxdb
[InfluxDB - TSDB engine] Negation operators do not work as expected
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Issue description:__
There is an issue on TSDB side with handling incoming `Flux` queries which contains negation operators in its body (`!=` or `!~`). Sometimes, the results of such queries execution are incorrect.
__Environment info:__
* System info: `Linux 4.15.0-65-generic x86_64`
* InfluxDB version: `v1.7.7`
__Config:__
Enable Flux:
```
# ...
[http]
# ...
flux-enabled = true
# ...
```
__Steps to reproduce:__
1. Start InfluxDB and write these points to the launched instance (`one measurement - different tags`):
```
curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE monitoring_main"
curl -i -XPOST 'http://localhost:8086/write?db=monitoring_main' --data-binary 'test_metric,host=tagA value=0.78'
curl -i -XPOST 'http://localhost:8086/write?db=monitoring_main' --data-binary 'test_metric,host=tagB value=0.64'
```
2. Execute this `Flux` query to validate that all series have been written successfully:
```
curl -XPOST -H "Content-Type:application/vnd.flux" -H "Accept:application/csv" -d 'from(bucket:"monitoring_main") |> range(start: -1500d) |> filter(fn: (r) => r._measurement == "test_metric")' http://localhost:8086/api/v2/query
```
The result should contain two rows with data:
```
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,dateTime:RFC3339,double,string,string,string
#group,false,false,true,true,false,false,true,true,true
#default,_result,,,,,,,,
,result,table,_start,_stop,_time,_value,_field,_measurement,host
,,0,2015-09-13T14:22:15.539309668Z,2019-10-22T14:22:15.539309668Z,2019-10-22T14:18:32.257030947Z,0.78,value,test_metric,tagA
,,1,2015-09-13T14:22:15.539309668Z,2019-10-22T14:22:15.539309668Z,2019-10-22T14:18:45.148127572Z,0.64,value,test_metric,tagB
```
3. Execute this `Flux` query:
```
curl -XPOST -H "Content-Type:application/vnd.flux" -H "Accept:application/csv" -d 'from(bucket:"monitoring_main") |> range(start: -1500d) |> filter(fn: (r) => r.host != "tagA")' http://localhost:8086/api/v2/query
```
__Expected behavior:__
The result of the query provided above should look like this output:
```
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,dateTime:RFC3339,double,string,string,string
#group,false,false,true,true,false,false,true,true,true
#default,_result,,,,,,,,
,result,table,_start,_stop,_time,_value,_field,_measurement,host
,,0,2015-09-13T14:28:05.121304894Z,2019-10-22T14:28:05.121304894Z,2019-10-22T14:27:04.20388464Z,0.64,value,test_metric,tagB
```
__Actual behavior:__
However, the result of the query provided above is `empty`.
___Note___:
After changing `filter` function's body from `fn: (r) => r.host != "tagA"` to `fn: (r) => r._measurement == "test_metric" and r.host != "tagA"` I've got the desired result:
```
curl -XPOST -H "Content-Type:application/vnd.flux" -H "Accept:application/csv" -d 'from(bucket:"monitoring_main") |> range(start: -1500d) |> filter(fn: (r) => r._measurement == "test_metric" and r.host != "tagA")' http://localhost:8086/api/v2/query
```
Result:
```
#datatype,string,long,dateTime:RFC3339,dateTime:RFC3339,dateTime:RFC3339,double,string,string,string
#group,false,false,true,true,false,false,true,true,true
#default,_result,,,,,,,,
,result,table,_start,_stop,_time,_value,_field,_measurement,host
,,0,2015-09-13T14:28:05.121304894Z,2019-10-22T14:28:05.121304894Z,2019-10-22T14:27:04.20388464Z,0.64,value,test_metric,tagB
```
Contributor guide
Research direction
Start by reproducing the standalone Flux filter query against the two points described in the issue, then compare it with the version that also filters on _measurement. Trace the TSDB query path handling != and !~; done when the standalone query returns only the tagB row and regression coverage verifies the behavior.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100