Unable to Test Session "with gap duration" Windowing
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Trying to write a unit test to verify the windowing behavior for session with gap duration, but my assumption is that there is a merging of IntervalWindows that normally happens that is not happening for my test pipeline, because my actual pipeline seems to work as expected, but my test fails.
I have been using these resources:
[http://www.waitingforcode.com/apache-beam/windows-apache-beam/read](http://www.waitingforcode.com/apache-beam/windows-apache-beam/read)
[https://beam.apache.org/blog/2016/10/20/test-stream.html](https://beam.apache.org/blog/2016/10/20/test-stream.html)
Here is an example of the issue:
```
String guid = "user1";
UniqueUserKey uniqueUser = makeUniqueUserKey(guid);
// the first value to
makePageView results in the timestamp: new Instant(value)
TimestampedValue>
homepage = makePageView(1, "HomePage", "homepage", guid, uniqueUser);
TimestampedValue> productDetails1 = makePageView(2, "Product Details", "product_details", guid, uniqueUser);
TestStream> testStream = TestStream.create(KvCoder.of(ProtoCoder.of(UniqueUserKey.class), ProtoCoder.of(PageLoadEvent.class)))
.addElements(homepage)
.addElements(productDetails1)
.advanceWatermarkTo(new Instant(8))
.advanceWatermarkToInfinity();
IntervalWindow window1 = new IntervalWindow(new Instant(1), new
Instant(3));
// This fails because productDetails1 is not in the window
PAssert.that(firstTransform).inFinalPane(window1).containsInAnyOrder(
homepage.getValue(),
productDetails1.getValue());
pipeline.run().waitUntilFinish();
```
Imported from Jira [BEAM-3755](https://issues.apache.org/jira/browse/BEAM-3755). Original Jira may contain additional context.
Reported by: jbfbell.
Contributor guide
Research direction
Start with the TestStream setup and the IntervalWindow and PAssert assertion shown in the issue; compare the event timestamps, watermark advances, and expected final pane. Reproduce the failing pipeline and determine whether the test or session-window merging behavior needs correction, with a passing assertion or documented expected result as the completion criterion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- stream-processing, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100