confluentinc / confluentinc/ksql

KSQL Stream Table Left join not finding match

Open
#2,133 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
315
Forks
1k
Avg merge
1d 7h
Merged PRs (30d)
36

Description

We have a stream and a table that both contain the same ITEM_ID ('66205') If we look in the stream we can find the item in a few seconds (setting the offset to beginning) querying the table takes a couple of **_minutes_** to find the match. The table is created over an existing topic using a CREATE TABLE statement (NOT a CTAS) both stream and table are partitioned and keyed the same way and the the rowtimes in the table are earlier that those in the stream.

When we do a left outer join between the stream and the table we get a response fairly quickly but the match is not detected.
I think there may be some timeout that if a match is not spotted with in X seconds it just emits the stream results.

Is this an accurate supposition and if so are there techniques to resolve this problem within KSQL

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the stream-table left outer join using an existing topic-backed CREATE TABLE, matching ITEM_ID, partitioning, keys, and rowtime ordering as described. Observe the lookup and join timing from the beginning of the stream, then determine whether the delayed table match is emitted; done means the behavior is explained and a documented resolution or confirmed defect is identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka, sql
Domain
databases, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.