influxdata / influxdata/influxdb

Explanation for timestamps in aggregated data

Open
#21,472 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

When a measurement's data is aggregated using `GROUP BY * , time(interval)` , I have observed the timestamps allocated is a little bizarre, I assume there is some rounding off mechanism behind this. What is the mechanism used allocate these timestamps ??

Example:
I started adding data in ascending order(numbers from 1 to 1336) to the measurement original at `2021-05-13T16:38:49.256462Z` , After around 4 to 4.5hrs I ran the following queries to get aggregated tables:

The original table: (Total 1336points)
```
> select * from original
name: original
time field1 field2 tag1
---- ------ ------ ----
2021-05-13T16:38:49.256462Z 0 0 Local
2021-05-13T16:38:59.561887Z 1 1 Static
2021-05-13T16:39:09.588163Z 2 2 Standard
2021-05-13T16:39:19.61457Z 3 3 Standard
2021-05-13T16:39:29.643384Z 4 4 Standard
.
.
2021-05-13T20:22:10.319583Z 1336 1336 Direct
```

Query1: select sum(field1) as field1_sum, first(field1) as field1_first, last(field1) as field1_last into "original_5min" from "original" where time > now() - 12w AND time < now() group by *,time(5m)

Resulting table:
```
> select * from original_5min
name: original_5min
time field1_first field1_last field1_sum tag1
---- ------------ ----------- ---------- ----
2021-05-13T16:35:00Z 7 7 7 Direct
2021-05-13T16:35:00Z 1 1 1 Static
2021-05-13T16:35:00Z 0 0 0 Local
2021-05-13T16:35:00Z 2 6 20 Standard
2021-05-13T16:40:00Z 11 34 174 Local
2021-05-13T16:40:00Z 10 36 146 Standard
.
.
2021-05-13T20:20:00Z 1324 1336 7982 Direct
```

Even though the first point was received at `2021-05-13T16:38:49.256462Z`, in the rolled up data the first point is at `2021-05-13T16:35:00Z`. This rounding off is understandable.

But for intervals like 30days and 365days, the timestamps given for the aggregated point is confusing, like the below example:

Query2: select sum(field1_sum) as field1_sum, first(field1_first) as field1_first, last(field1_last) as field1_last into "original_1months" from "original_5min" where time > now() - 12w AND time < now() group by *,time(30d)

Resulting table:
```
> select * from "original_1months"
name: original_1months
time field1_first field1_last field1_sum tag1
---- ------------ ----------- ---------- ----
2021-05-03T00:00:00Z 7 1336 231941 Direct
2021-05-03T00:00:00Z 1 1338 232027 Static
2021-05-03T00:00:00Z 2 1339 224724 Standard
2021-05-03T00:00:00Z 0 1340 209778 Local
```

Why in the table "original_1months" the timestamp of the first point is `2021-05-03T00:00:00Z `?? Why it is being pointed to 3rd May's date ?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the GROUP BY *,time(5m) and GROUP BY *,time(30d) queries from the issue and inspect the resulting timestamps. Trace the timestamp-alignment behavior for grouped intervals, then document why the 30-day result uses 2021-05-03T00:00:00Z and how these boundaries are selected.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.