influxdata / influxdata/influxdb
Release 2.2 out-of-memory when using onKeep in filter on minimal query
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
With Influx 2.2, the following minimal query generates 100% CPU and RAM leading to OOM.
Same query is running fine if I downgrade to 2.1. Tried also with system with 64GB RAM.
__Steps to reproduce:__
Run this query in 2.2 (it completely freezes my system, so you are warned)
```sh
import "experimental/array"
table=array.from(rows: [{_measurement: "m0", myvalue1: 0, myvalue2: 1, _time: 2020-01-01T00:00:00Z,},
{_measurement: "m0", myvalue1: 0, myvalue2: 1, _time: 2020-01-02T00:00:00Z}])
check = table
|> filter(fn: (r) => r.myvalue1 >= r.myvalue2, onEmpty: "keep" )
|> sort(columns: ["myvalue2"], desc: true)
|> limit(n: 1)
|> yield(name:"check")
```
__Expected behavior:__
Query generates no results
__Actual behavior:__
Query generates 100% CPU and RAM leading to OOM.
Query runs fine (no results) If I remove `onEmpty:"keep"` in the filter.
Alternatively I can leave the filter unchanged and remove the ` |> limit(n: 1)` to also avoid OOM (not what I want)
Same query (and all the other combinations) run fine if I downgrade to 2.1.
__Environment info:__
* System info: Linux 5.4.0-109-generic x86_64
* InfluxDB version: InfluxDB v2.2.0 (git: a2f8538837) build_date: 2022-04-06T17:36:40Z
__Config:__
```sh
bolt-path = "/var/lib/influxdb/influxd.bolt"
engine-path = "/var/lib/influxdb/engine"
http-bind-address = ":8086"
log-level = "error"
query-concurrency = 10
query-queue-size = 10
reporting-disabled = true
secret-store = "bolt"
session-length = 60
storage-retention-check-interval = "30m0s"
tls-cert = "/etc/ssl/influxdb-selfsigned.crt"
tls-key = "/etc/ssl/influxdb-selfsigned.key"
tls-min-version = "1.1"
tls-strict-ciphers = true
```
__Performance:__
Could not generate profiles during bug because system is unusable due to 100% CPU and RAM.
Contributor guide
Research direction
Start by reproducing the supplied Flux query on InfluxDB 2.2, then compare its behavior with 2.1 and with the stated filter and limit variations. Done means the query returns no results without exhausting CPU or memory; the issue names no source files or tests to begin with.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100