influxdata / influxdata/influxdb

Monolith OSS - Limit query consumes high CPU usage for look backs 12+hours

Open
#25,573 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/perf v3
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

### Problem 1
The SQL limit query consumes very high CPU utilization even for a single series. On a `m6dg.large` machine which is of 2 cores and 8 GiB RAM, we observe that the query uses more than 75% and in many cases nearly 95%.

### Problem 2
As the LIMIT increases, drastic increase in CPU usage is observed. LIMIT 1 -> LIMIT 5.

Note: Happens only in OSS. This has an effect on the latency as well and these experiments are without concurrency.

### Configuration
The problem was found with the below basic setup,

- 2 cores, 8 GiB RAM
- Light Ingest, about 17 rows/second
- Executed as a stand-alone query (tried as part of a suite as well, same concern)
- Query look back used was 12+ hours (approx. 734400 rows scanned)

### Queries

> SELECT * FROM senec_data WHERE device_id = '10' and time > (now() - interval'{{.QueryRange}} minute')::timestamp ORDER BY time DESC LIMIT 1;
>
> SELECT * FROM senec_data where device_id = '10' and time > (now() - interval'{{.QueryRange}} minute')::timestamp order by time desc limit 5;

### Metrics/Evidence
Q5 query - LIMIT 5 reaching 93.5%
image

F1 query - LIMIT 1 reaching 74.4%
image

Latency over 100 runs
image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source files or tests are named. Start by reproducing the two SQL queries with the reported 12+ hour lookback and approximately 734,400 scanned rows, comparing LIMIT 1 and LIMIT 5 on the described setup. Done means identifying and addressing the OSS CPU and latency increase, but the issue provides no target threshold.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.