influxdata / influxdata/influxdb
Flux csv query causes InfluxDB 2.0 to consume all CPU
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
Run this query:
```
import "csv"
csvData = "
#datatype,string,long,string,long,dateTime:RFC3339
#group,false,false,false,false,false
#default,,,,,
,result,table,env,_value,_time
,,0,foo,3,2018-05-08T20:50:00Z
,,0,bar,4,2018-05-08T20:50:01Z
,,0,baz,6,2018-05-08T20:50:02Z
"
csv.from(csv: csvData)
|> range(start: -100000d)
|> aggregateWindow(every: 1m, fn: max)
```
__Expected behavior:__
Describe what you expected to happen.
A result or error should return immediately
__Actual behavior:__
Describe What actually happened.
Query hangs, `influxd` process consumes all CPU cores.
__Environment info:__
* System info: Darwin 18.7.0 x86_64
* InfluxDB version: https://github.com/influxdata/influxdb/commits/9495ec1c4e
* Other relevant environment details: built with `make`
__Config:__
Copy any non-default config values here or attach the full config as a gist or file.
Nothing
__Logs:__
Include snippet of errors in log.
Nothing of note.
__Performance:__
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.
[iostat.txt](https://github.com/influxdata/influxdb/files/4490026/iostat.txt)
[vars.txt](https://github.com/influxdata/influxdb/files/4490027/vars.txt)
[profiles.tar.gz](https://github.com/influxdata/influxdb/files/4490028/profiles.tar.gz)
Contributor guide
Research direction
Run the supplied Flux query against the reported InfluxDB revision and inspect profiles.tar.gz, iostat.txt, and vars.txt for the source of the CPU loop. Trace the aggregateWindow path for CSV input and verify that the query returns promptly without consuming all CPU cores.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100