confluentinc / confluentinc/ksql
KSQL Stream Table Left join not finding match
- Dominant language
- Java
- Stars
- 315
- Forks
- 1k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 36
Description
We have a stream and a table that both contain the same ITEM_ID ('66205') If we look in the stream we can find the item in a few seconds (setting the offset to beginning) querying the table takes a couple of **_minutes_** to find the match. The table is created over an existing topic using a CREATE TABLE statement (NOT a CTAS) both stream and table are partitioned and keyed the same way and the the rowtimes in the table are earlier that those in the stream.
When we do a left outer join between the stream and the table we get a response fairly quickly but the match is not detected.
I think there may be some timeout that if a match is not spotted with in X seconds it just emits the stream results.
Is this an accurate supposition and if so are there techniques to resolve this problem within KSQL
Contributor guide
Research direction
Start by reproducing the stream-table left outer join using an existing topic-backed CREATE TABLE, matching ITEM_ID, partitioning, keys, and rowtime ordering as described. Observe the lookup and join timing from the beginning of the stream, then determine whether the delayed table match is emitted; done means the behavior is explained and a documented resolution or confirmed defect is identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, sql
- Domain
- databases, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100