influxdata / influxdata/influxdb

Slowness when doing basic queries

Open
#23,715 0 comments 3 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

### Discussed in https://github.com/influxdata/influxdb/discussions/23690

Originally posted by **jeremyaouad** August 31, 2022
I was testing with Influx 2.3 container.
I have a set of 80 metrics, emitted every 10 seconds, for about 500 processes => cardinality of 500*80 = 40000.
I didn't put any limits on the influxdb container cpu and memory.
The machine I'm testing on has 28 cores and 500GB ram. I'm using the ram as storage for influxdb (bbolt mounted on a tmpfs volume), with a read/write speed of 3 GB/sec.
I wrote 12 very basic queries of the sort "select * from x where y = z" in a file and launched them periodically, every 10 sec, and timed the duration of the execution. In the query, I set the range for the past 10 seconds. The returned data set would be between 200 and 400 records.

I observed the following:
- For the write performance, the cpu went up to 5-10% max. Memory was also good.
- For the read performance, whenever the script was launched, the cpu usage went up to 12 cores, memory was ok. However, the execution time kept on increasing from a couple of seconds to 2 min as time passed. During the execution time, the cpu usage would be 12 cores.
- What puzzled me more is the following: I stopped emitting metrics in influxdb, I queried for the past 10 seconds, and I was still getting a 2 min query time.

I also launched the queries with the profiler output. The execution time was taking all of the time. Specifically this:
influxdb.readWindowAggregateSource,ReadWindowAggregateByTime11

Is there something I'm missing?

Thanks.

Contributor guide

Open the contributing guide

Research direction

Reproduce the report using the InfluxDB 2.3 container, 40,000-series cardinality, and repeated 10-second queries, then inspect the profiler output around influxdb.readWindowAggregateSource and ReadWindowAggregateByTime11. Done should include an identified cause for the increasing query time and evidence that the basic queries no longer take minutes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.