influxdata / influxdata/influxdb
Explanation for timestamps in aggregated data
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
When a measurement's data is aggregated using `GROUP BY * , time(interval)` , I have observed the timestamps allocated is a little bizarre, I assume there is some rounding off mechanism behind this. What is the mechanism used allocate these timestamps ??
Example:
I started adding data in ascending order(numbers from 1 to 1336) to the measurement original at `2021-05-13T16:38:49.256462Z` , After around 4 to 4.5hrs I ran the following queries to get aggregated tables:
The original table: (Total 1336points)
```
> select * from original
name: original
time field1 field2 tag1
---- ------ ------ ----
2021-05-13T16:38:49.256462Z 0 0 Local
2021-05-13T16:38:59.561887Z 1 1 Static
2021-05-13T16:39:09.588163Z 2 2 Standard
2021-05-13T16:39:19.61457Z 3 3 Standard
2021-05-13T16:39:29.643384Z 4 4 Standard
.
.
2021-05-13T20:22:10.319583Z 1336 1336 Direct
```
Query1: select sum(field1) as field1_sum, first(field1) as field1_first, last(field1) as field1_last into "original_5min" from "original" where time > now() - 12w AND time < now() group by *,time(5m)
Resulting table:
```
> select * from original_5min
name: original_5min
time field1_first field1_last field1_sum tag1
---- ------------ ----------- ---------- ----
2021-05-13T16:35:00Z 7 7 7 Direct
2021-05-13T16:35:00Z 1 1 1 Static
2021-05-13T16:35:00Z 0 0 0 Local
2021-05-13T16:35:00Z 2 6 20 Standard
2021-05-13T16:40:00Z 11 34 174 Local
2021-05-13T16:40:00Z 10 36 146 Standard
.
.
2021-05-13T20:20:00Z 1324 1336 7982 Direct
```
Even though the first point was received at `2021-05-13T16:38:49.256462Z`, in the rolled up data the first point is at `2021-05-13T16:35:00Z`. This rounding off is understandable.
But for intervals like 30days and 365days, the timestamps given for the aggregated point is confusing, like the below example:
Query2: select sum(field1_sum) as field1_sum, first(field1_first) as field1_first, last(field1_last) as field1_last into "original_1months" from "original_5min" where time > now() - 12w AND time < now() group by *,time(30d)
Resulting table:
```
> select * from "original_1months"
name: original_1months
time field1_first field1_last field1_sum tag1
---- ------------ ----------- ---------- ----
2021-05-03T00:00:00Z 7 1336 231941 Direct
2021-05-03T00:00:00Z 1 1338 232027 Static
2021-05-03T00:00:00Z 2 1339 224724 Standard
2021-05-03T00:00:00Z 0 1340 209778 Local
```
Why in the table "original_1months" the timestamp of the first point is `2021-05-03T00:00:00Z `?? Why it is being pointed to 3rd May's date ?
Contributor guide
Research direction
Start by reproducing the GROUP BY *,time(5m) and GROUP BY *,time(30d) queries from the issue and inspect the resulting timestamps. Trace the timestamp-alignment behavior for grouped intervals, then document why the 30-day result uses 2021-05-03T00:00:00Z and how these boundaries are selected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- databases
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100