influxdata / influxdata/influxdb

Negation operators don't work as expected for meta queries

Open
#9,976 5 comments 0 reactions 0 assignees View on GitHub
1.x area/queries kind/bug
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

The negation operators (`!=` and `!~`) don't work as expected for some meta queries. The following meta queries are not affected:

```
SHOW SERIES
SHOW TAG VALUES
SHOW FIELD KEYS
```

Here are examples of affected meta queries:

#### Example 1 - SHOW MEASUREMENTS

This example fails on `tip` and also on `1.4.2` and `1.3.6`.
```
> create database db
> use db
Using database db
>
> insert cpu,region=west value=1
> insert cpu,region=east value=1
>
> show measurements
name: measurements
name
----
cpu
>
> show measurements where region = 'west'
name: measurements
name
----
cpu
>
> show measurements where region = 'east'
name: measurements
name
----
cpu
>
> show measurements where region != 'east'
>
> show measurements where region != 'west'
>
```

In both of these cases we should expect the `cpu` measurement to be returned. The issue here is that because there exist a series in the measurement that doesn't satisfy `!= 'east'`, the entire measurement is excluded from the result set. To clarify this a little further:

```
> create database db
> use db
Using database db
>
> insert cpu,region=east value=1
> insert cpu,region=west value=1
> insert mem,region=east value=1
>
> show measurements where region = 'east'
name: measurements
name
----
cpu
mem
>
> show measurements where region != 'west'
name: measurements
name
----
mem
>
>
```
The `!= 'west'` returns `mem` because `mem` does not have a series where `region = west`.

#### Example 2 - SHOW TAG KEYS

```
> drop database db
> create database db
>
> insert cpu,region=west value=1
> insert cpu,region=east value=1
>
> show tag keys
name: cpu
tagKey
------
region
>
> show tag keys where region != 'east'
> show tag keys where region !~ 'west'
ERR: error parsing query: found west, expected regex at line 1, char 30
> show tag keys where region !~ /west/
```

The same problem as `SHOW MEASUREMENTS`. The negation options will exclude any tag key if there is an example series where the predicate is not true.

The issue is to do with the way the [tags are matched](https://github.com/influxdata/influxdb/blob/master/tsdb/index.go#L1310-L1434) when looking for measurements to return.

(For those with access, there was an internal conversation here: https://influxdb.slack.com/archives/C03ARPJ6W/p1510833917000363 back in November.)

Contributor guide

Open the contributing guide

Research direction

Start in tsdb/index.go, particularly the tag-matching logic around lines 1310-1434, and reproduce the SHOW MEASUREMENTS and SHOW TAG KEYS examples from the issue. Trace how != and !~ predicates are applied across series; done means affected meta queries return measurements and tag keys according to the demonstrated negation semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.