influxdata / influxdata/influxdb

Spikes when using sum() grouped by time interval

Open
#23,985 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Dear all,

I see spikes when using sum() over time-span > 7d when grouped by time interval.

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. create a time series with hourly cumulative data (and some null's for missing data) for e.g. 30 days, e.g. using the data from my influxdb session over past 30 days.
`select * from "Gas-usage" where time>=now()-30d and time <=now() `
[influx-raw-30d.txt](https://github.com/influxdata/influxdb/files/10244042/influx-raw-30d.txt)

2. use the query like this using sum() grouped by time interval:
`SELECT sum("value") FROM "Gas-usage" WHERE time >= now() - 30d and time <= now() GROUP BY time(1h) fill(null)`
3. observe a number of spikes in the data

__Expected behavior:__
Here is the same query when using mean() instead of sum(), mean() should be similar when time interval is 1 hour:
![image](https://user-images.githubusercontent.com/6523124/208054107-56974283-5ae9-4468-90fb-4c3b10d238a9.png)
![image](https://user-images.githubusercontent.com/6523124/208054208-b53eb27d-6948-4005-89f1-de153f6d13d3.png)

__Actual behavior:__
I see spikes in larger queries that use a sum() in (see also #7647). I use Grafana for display, but also get the same results when running the query directly in influxdb. Here are some graphs that show the spikes over cumulative gas usage (hourly measurements that are null when value didn't change) that is always increasing. The spikes seem to have a value that is multiple of the cumulative value. The issue seems to appear when the time-range is longer than 7 days (here 30 days):
![image](https://user-images.githubusercontent.com/6523124/208053760-e55d99da-5476-4744-b493-9dbf149cd7b3.png)
![image](https://user-images.githubusercontent.com/6523124/208053979-c8855e93-b531-45ee-b3da-d00594cb197e.png)
A lot more spikes when time-span is larger (e.g. for 50 days):
![image](https://user-images.githubusercontent.com/6523124/208063171-59be68c1-b4ac-4dc4-9d71-0bedd91fd8cb.png)

__Environment info:__
influxdb 1.8.10 on ubuntu Linux
* System info: influxdb 1.8.10 on ubuntu Linux
* InfluxDB version: InfluxDB v1.8.10 (git: 1.8 688e697c51fd)
* Other relevant environment details: see also #7647 for similar issue back in 2016

I want to use the sum since to show the gas usage per time interval at different time-zoom levels, with mean this works when time-interval is 1 hour, but not when it is larger (then I get an average instead of a sum).
![image](https://user-images.githubusercontent.com/6523124/208054792-53c54346-e6a3-4adb-b1b6-bb2c8dfba470.png)
![image](https://user-images.githubusercontent.com/6523124/208054939-d094ec48-055b-4936-ab0c-c6ca1743bea5.png)
![image](https://user-images.githubusercontent.com/6523124/208055146-bc66ef12-dd78-4e60-809c-86ac620ef98f.png)
![image](https://user-images.githubusercontent.com/6523124/208055239-2ce79c8e-54d5-460f-86b9-ef8408971125.png)

Here are CSV files for the data of the first two graphs using sum() and mean().
- [Gas-data-2022-12-16 09 28 50-using-mean.csv](https://github.com/influxdata/influxdb/files/10243972/Gas-data-2022-12-16.09.28.50-using-mean.csv)
- [Gas-data-2022-12-16 09 28 21-using-sum.csv](https://github.com/influxdata/influxdb/files/10243974/Gas-data-2022-12-16.09.28.21-using-sum.csv)

__Config:__
Copy any non-default config values here or attach the full config as a gist or file.

__Logs:__
Include snippet of errors in log.

__Performance:__
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.

```sh
# Commands should be run when the bug is actively happening.
# Note: This command will run for ~30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=30s"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz` and `iostat.txt` output files.
```
Kind regards,
Dennis

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported SELECT sum query on InfluxDB 1.8.10 with the attached hourly data, then compare its grouped output with mean() and the supplied CSV files. No source file or test is named; done would require identifying the cause of the long-range spikes and adding a regression test showing correct results for grouped sum queries.

Written by the indexing model from the issue text.

Assessment

Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.