influxdata / influxdata/influxdb
The contains function may not be optimized
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
### Environment info:
**influxDB version**: 2.0.3
**System info**: from docker
Debain, X86_64, 8-core Intel(R) Xeon(R) Platinum 8260 CPU @ 2.40GHz, 16GB RAM
### Data describe:
**BucketName**: 15day_profile_bucket
**MeasurementName**: function_info
**Tags**: Function, Pid, Tid, ProcessName, UUID, State
**Fields**: Internal, cumulative
there may be 2.4w record and 200 series in 1 minute.
### Problem:
The `contains` function query is very slow, it seems that the group key filter is not used.
The following flux query took 0.63s:
```
from(bucket: "15day_profile_bucket")
|> range(start: 2021-03-03T05:54:39.611Z, stop: 2021-03-03T06:54:39.611Z)
|> filter(fn: (r) => r["_measurement"] == "function_info" )
|> limit(n: 1)
|> filter(fn: (r) => contains(value: r["UUID"], set: ["7f0a1436-37ad-4b7a-9ab1-7acce9ee3060"]) )
|> yield()
```
this is image:

but the flux query took 37.88s:
```
from(bucket: "15day_profile_bucket")
|> range(start: 2021-03-03T05:54:39.611Z, stop: 2021-03-03T06:54:39.611Z)
|> filter(fn: (r) => r["_measurement"] == "function_info" )
|> filter(fn: (r) => contains(value: r["UUID"], set: ["7f0a1436-37ad-4b7a-9ab1-7acce9ee3060"]) )
|> limit(n: 1)
|> yield()
```
this is image:

### Expected behavior:
The time spent on the two queries differs too much.
Because UUID is a tag field, so the first flux query ( is filter first then limit), and the second query (is limit first then filter) should no big difference.
So I guess the `contains` function does not use the group key for filtering, but scans all the data。
### Use Case:
Our team used influxdb-v2, but that is the bottleneck of our project now.
I have tried to use multiple `or` operations to replace `contains` function, but when the number of filters is large(70+), the or operation is slower.
Contributor guide
Research direction
Start by running the two Flux queries in the issue and comparing their execution times with the same tag-filtering data. Investigate whether contains uses the UUID group key before scanning rows; done means the two query forms no longer show the reported performance gap, with a regression test covering the behavior.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100