confluentinc / confluentinc/ksql

ksqlDB auto close windows

Open
#9,840 2 comments 0 reactions 0 assignees View on GitHub
enhancement requires-streams-change streaming-engine
Dominant language
Java
Stars
315
Forks
1k
Avg merge
1d 7h
Merged PRs (30d)
36

Description

Good afternoon. The problem is very simple, when you use **window functions (TUMBLING**) with **emit final**, then the window is closed only when the next message arrives, and it can come in 5 hours and the client will get the message after 5 hours instead of the 10 seconds delay.
The current implementation needs to be improved. Include in ksqldb internal mechanisms to close windows automatically

![image](https://user-images.githubusercontent.com/61819835/226478055-dc4a123c-4397-4eb0-b6ed-1e185b6fab76.png)

```
SELECT userId, count(*),
from_unixtime(WINDOWSTART) as Window_Start,
from_unixtime(WINDOWEND) as Window_End,
from_unixtime(max(ROWTIME)) as Window_Emit
FROM s_t1
WINDOW TUMBLING (SIZE 10 SECONDS)
GROUP BY userId
EMIT FINAL;
```

We have a large company and wanted to use your technology, but this problem changes everything.
@mjsax @big-andy-coates @JimGalasyn @vcrfxia

Contributor guide

Open the contributing guide

Research direction

Start with the reported TUMBLING window query using EMIT FINAL and trace the ksqlDB windowing implementation; the payload does not name a source file or test. Done means a 10-second window closes and emits its final result without waiting for another message, including when the next message arrives hours later.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases, stream-processing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.