influxdata / influxdata/kapacitor

Barrier node does not flush last window with groupBy

Open
#2,244 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

In the following streaming task, the barrier node will not flush the last window of data through the pipeline:

```
stream
|from()
.measurement(measurement)
|groupBy(tags)
|barrier()
.idle(30s)
|window()
.period(1m)
.every(1m)
.align()
```

However if we replace the `|groupBy` node with the `.groupBy` property on the `|from` node like so:

```
stream
|from()
.measurement(measurement)
.groupBy(tags)
|barrier()
.idle(30s)
|window()
.period(1m)
.every(1m)
.align()
```

the barrier node does indeed flush the last window through the pipeline.

Contributor guide

Open the contributing guide

Research direction

Reproduce the streaming task shown in the issue, first with groupBy as its own node and then with groupBy on from(). Compare how barrier() and window() handle the final window in each case. Done means the standalone groupBy form flushes the last window through the pipeline consistently with the from().groupBy form, with a regression test covering both cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.