influxdata / influxdata/influxdb
applying aggregateWindow twice results in out-of-order and duplicate results
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.
You should be able to use to following, kind of minimal example. Note that this example by itself does not make any sense, as it is reduced to a very simple example. At the same time, this short example is good to show that something is wrong here.
```
import "influxdata/influxdb/sample"
import "date"
Monday = date.add(d: 4d, to: date.truncate(t: date.sub(d: 4d, from: now()), unit: 1w))
sample.data(set: "airSensor")
|> range(start: date.sub(d:9d, from: Monday),
stop: Monday)
|> aggregateWindow(every: 1h, fn: last)
|> aggregateWindow(every: 1d, fn: last)
```
__Expected behavior:__
I expect the first aggregateWindow to bin results into 1 hour intervals, using the last value in that bin, and the second aggregateWindow to do the same but daily, and using the bin values from the first aggregateWindow as input.
__Actual behavior:__
This is what I get as the first rows:
```
0 airSensors co 2022-11-27T00:00:00.000Z TLM0100
0 airSensors co 2022-12-05T00:00:00.000Z TLM0100
0 airSensors co 2022-11-28T00:00:00.000Z TLM0100
0 airSensors co 2022-11-29T00:00:00.000Z TLM0100
0 airSensors co 2022-11-30T00:00:00.000Z TLM0100
0 airSensors co 2022-12-01T00:00:00.000Z TLM0100
0 airSensors co 2022-12-02T00:00:00.000Z TLM0100
0 airSensors co 2022-12-03T00:00:00.000Z TLM0100
0 airSensors co 2022-12-04T00:00:00.000Z TLM0100
0 airSensors co 2022-12-05T00:00:00.000Z TLM0100
```
Note the time stamp of the second row. Not only is it out of order with respect to time, it is also duplicated at the end. Also, not visible in this example, on my data, the values can actually differ (slightly) for the two duplicates (timestamp is the same, but the measurement value is not).
I am not sure if this is related to #21111, but since there it is the first value that is wrong and here the second, I rather open a new issue and point to possibly related ones.
__Environment info:__
* System info: Linux 5.15.76-v8+ aarch64
* InfluxDB version: InfluxDB v2.5.1 (git: 5b6fdbf05d) build_date: 2022-11-02T18:06:20Z
* Other relevant environment details: Debian package
Contributor guide
Research direction
Start by running the supplied Flux query against InfluxDB v2.5.1 and inspect the ordering and duplicate timestamps in its output. Compare the behavior with issue #21111, which the report identifies as possibly related. Done means repeated aggregateWindow calls preserve daily result ordering and do not emit duplicate timestamps, with a regression test for the example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100