influxdata / influxdata/influxdb

aggregateWindow first window value - same as last in data series

Open
#21,111 2 comments 1 reaction 0 assignees View on GitHub
area/2.x area/flux area/queries
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Hi I have seen a strange behaviour performed by aggregateWindow - with this query.

from(bucket: "GH-0013-210119-01")
|> range(start: 2020-08-19T00:00:00Z, stop: 2020-08-20T00:00:00Z)
|> filter(fn: (r) => (r["_measurement"] == "history"))
|> filter(fn: (r) => (r["_field"] == "val"))
|> filter(fn: (r) => (r["tag_name"] == "CurrentSetpoint"))
//|> aggregateWindow(every: 1s, fn: mean, createEmpty: true)
//|> fill(column: "_value", usePrevious: true)
//|> aggregateWindow(every: 5m, fn: mean, createEmpty: true)
|> yield()

When running the above query (including the comments) I get the raw values from influxdb (reduced from a PostgreSql database, that contains values for each second, but only when a significant change is observed, a value can be found in influxdb):
Time val
2020-08-19T00:00:00Z 0
2020-08-19T05:09:57Z 1
2020-08-19T05:09:58Z 2.1
2020-08-19T05:09:59Z 3.1
2020-08-19T05:10:00Z 4.1

When applying the first aggregate window for every second and doing a fill with the previous value, the data are as expected (and a one second lag due to the window function, that has on importance for my application):

2020-08-19T00:00:01Z 0
2020-08-19T00:00:02Z 0
... a lot of zero's
2020-08-19T05:09:58Z 1
2020-08-19T05:09:59Z 2.1
2020-08-19T05:09:59Z 3.1
2020-08-19T05:10:00Z 4.1

Then the big surprice applying a second aggregateWindow(every: 5m, fn: mean, createEmpty: true) on the above data gives:

2020-08-19T00:00:00Z 200 !!!
2020-08-19T00:05:00Z 0
.... some zero's
2020-08-19T05:10:00Z 0.0103
2020-08-19T05:15:00Z 0.0410

The value 200 matches (in this case) the last value in the series and if I change the stop time, the value changes often accordingly to the new last value. As a matter of fact, the value is always the same value - despite the function used.

I'm using the Data Explorer and looking at the raw values

__Expected behavior:__
First value should be calculated according to window, values and function used

__Actual behavior:__
First value is often the same as the last value in the resulting data series - but not the correct value anyway.

__Environment info:__
* System info: Linux 5.4.0-66-generic x86_64
* InfluxDB version: InfluxDB 2.0.4 (git: 4e7a59bb9a) build_date: 2021-02-08T17:47:02Z

__Config:__
* Plain installation as Debian package

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the supplied query in Data Explorer, including both aggregateWindow stages and the fill step. Compare the first five-minute window with the expected values while varying the aggregation function and stop time; done means the first window reflects its own data rather than the series' final value.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.