influxdata / influxdata/kapacitor
FR: WindowNode should be able to emit empty aligned windows rather than just skipping the empty ones
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
Hi,
Currently the following will output nothing for days with no data-points:
```
stream
|from()
.measurement('test')
|window()
.align()
.period(1d)
.every(1d)
|count('x')
```
when sending this (where t2 = t1 plus 2 days):
```
test x=1 t1
test x=1 t2
```
it outputs t1 and t2's days with counts 1, but skips the day between them (which had no data-points)
Sometimes we want 0 count for the days when we received no data-points at all, so that further stats (e.g. last x days rolling mean, etc.. can be accurate).
Is there any way of doing it in the current release? (apart from having a "null" data-point every day in the input measurement which we filter out from the aggregations after, i.e. decrement the count)
If there isn't, it could be an improvement on the WindowNode (I guess, only when it's aligned, for non-aligned windows the emit time depends on the first data point of the window):
```
|window()
.align()
.period(1d)
.every(1d)
.emitEmpty() // this would work only when .align()-ed
```
The implementation could be something like: in window.go, in the Insert(Point) function, when the data-point is after nextEmit, it should keep incrementing nextEmit and emit the empty windows (returning an array of batches instead of just one) until the point is before the nextEmit.
(assuming that the aggregating functions can deal with empty windows of course).
Thanks!
Contributor guide
Research direction
Start in window.go's Insert(Point) implementation and trace how aligned windows advance when a point arrives after nextEmit. Check how batches and aggregating functions represent empty windows, then verify the requested behavior with the two-point, three-day example while preserving the stated non-aligned-window behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100