[STORM-3818] Joining more than 2 streams in Stream API
- Dominant language
- Java
- Stars
- 6.7k
- Forks
- 4k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 62
Description
Joining two streams in storms Stream API works fine as described in the docs with the scheme: streamA.window(...).join(streamB)
However, when joining the result with a third join, a `punctuation error` is thrown.
So there is a way missing of joining three streams? I think, that one window configuration is enough. So the pseudocode would look like: streamA.window(...).join(streamB).join(streamC). However, this raises a punctuation error.
Here is the error log:
Caused by: java.lang.IllegalStateException: Received punctuation from streams [s6] expected [s7]
at org.apache.storm.streams.ProcessorBoltDelegate.shouldPunctuate(ProcessorBoltDelegate.java:287) ~[classes/:?]
at org.apache.storm.streams.ProcessorBoltDelegate.punctuateInitialProcessors(ProcessorBoltDelegate.java:189) ~[classes/:?]
at org.apache.storm.streams.ProcessorBoltDelegate.process(ProcessorBoltDelegate.java:179) ~[classes/:?]
at org.apache.storm.streams.WindowedProcessorBolt.execute(WindowedProcessorBolt.java:68) ~[classes/:?]
at org.apache.storm.topology.WindowedBoltExecutor.boltExecute(WindowedBoltExecutor.java:371) ~[classes/:?]
at org.apache.storm.topology.WindowedBoltExecutor$1.onActivation(WindowedBoltExecutor.java:364) ~[classes/:?]
at org.apache.storm.windowing.WindowManager.onTrigger(WindowManager.java:156) ~[classes/:?]
at org.apache.storm.windowing.TimeTriggerPolicy$1.run(TimeTriggerPolicy.java:119) ~[classes/:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
... 1 more
---
Originally reported by Heinrich, imported from: Joining more than 2 streams in Stream API
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the punctuation handling in ProcessorBoltDelegate, especially shouldPunctuate and punctuateInitialProcessors, then follow the WindowedProcessorBolt call path shown in the stack trace. Reproduce a chained three-stream join based on the documented two-stream Stream API example. Done means joining streamA, streamB, and streamC with one window no longer raises the reported punctuation error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100