influxdata / influxdata/kapacitor
Use right time boundary instead of left in Kapacitor/InfluxDB
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
Consider the following InfluxDB query
```sql
SELECT mean(*) FROM ... time >= '2017-03-13T17:50:00Z' AND time < '2017-03-13T17:51:00Z'
```
We will refer to **`'2017-03-13T17:50:00Z' `** as the left boundary, lower boundary, or `tMin`.
Similarly, we will refer to **` '2017-03-13T17:51:00Z'`** as the right boundary, upper boundary, or `tMax`.
Executing this query against InfluxDB will yield the following result
```
time mean
---- ----
2017-03-13T22:29:00Z 8.083532716666666e+08
```
Note here that this is the **lower boundary** for the time interval.
A similar operation in Kapacitor
```
batch
|query('SELECT queryDurationNs FROM "_internal".monitor.queryExecutor')
.period(1m)
.every(1m)
.align()
|mean('queryDurationNs')
```
would result in
```
time mean
---- ----
2017-03-13T17:51:00Z 8.083532716666666e+08
```
Note here that this is the **upper boundary** for the time interval.
InfluxDB and Kapacitor choose different time boundaries. Kapacitor chooses `tMax` and InfluxDB chooses `tMin`. The choice between using `tMax` or `tMin` is somewhat arbitrary for InfluxDB, however the same thing cannot be said for Kapacitor.
This is because of Kapacitors ability to do complex joining operations on overlapping time windows. Meaning if you were to join the mean over the last month with the the mean over the least day, you would need their resulting values to occur at the same time. Using the most recent time, `tMax`, to represent both points makes sense.
This has lead to two issues in the community: [issue](https://community.influxdata.com/t/influxdb-cq-vs-kapacitor-batch-vs-kapacitor-stream/202) and https://github.com/influxdata/kapacitor/issues/1257
At the moment there are two ways to go about fixing the issue
1. The `/query` endpoint on InfluxDB can take a query parameter that specifies which time boundary should be used.
Similar issues with right and left handed ness of time buckets has come up else where in InfluxDB and therefore it may be worthwhile to implement it there.
https://github.com/influxdata/influxdb/issues/7738
https://github.com/influxdata/influxdb/issues/7752
2. The discrepancy on timestamps is only an issue for some types of queries. Kapacitor should be able to detect if a supplied query meets these criteria and if it does, modify the timestamps of the data as needed.
Contributor guide
Research direction
Start by reproducing the contrasting InfluxDB query and Kapacitor batch results described in the issue, then inspect the related InfluxDB issues and Kapacitor issue #1257. The scope is not settled: done requires an agreed approach and consistent time-boundary behavior, with validation that overlapping windows produce compatible timestamps.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- data, stream-processing
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100