apache / apache/beam

Unable to Test Session "with gap duration" Windowing

Open
#18,630 0 comments 0 reactions 0 assignees View on GitHub
bug P3 tests
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

Trying to write a unit test to verify the windowing behavior for session with gap duration, but my assumption is that there is a merging of IntervalWindows that normally happens that is not happening for my test pipeline, because my actual pipeline seems to work as expected, but my test fails.  

I have been using these resources:

[http://www.waitingforcode.com/apache-beam/windows-apache-beam/read](http://www.waitingforcode.com/apache-beam/windows-apache-beam/read)

[https://beam.apache.org/blog/2016/10/20/test-stream.html](https://beam.apache.org/blog/2016/10/20/test-stream.html)

 

Here is an example of the issue:
```

String guid = "user1";
UniqueUserKey uniqueUser = makeUniqueUserKey(guid);
// the first value to
makePageView results in the timestamp: new Instant(value)
TimestampedValue>
homepage = makePageView(1, "HomePage", "homepage", guid, uniqueUser);
TimestampedValue> productDetails1 = makePageView(2, "Product Details", "product_details", guid, uniqueUser);

TestStream> testStream = TestStream.create(KvCoder.of(ProtoCoder.of(UniqueUserKey.class), ProtoCoder.of(PageLoadEvent.class)))

.addElements(homepage)
.addElements(productDetails1)
.advanceWatermarkTo(new Instant(8))

.advanceWatermarkToInfinity();

IntervalWindow window1 = new IntervalWindow(new Instant(1), new
Instant(3));

// This fails because productDetails1 is not in the window
PAssert.that(firstTransform).inFinalPane(window1).containsInAnyOrder(

homepage.getValue(),
productDetails1.getValue());

pipeline.run().waitUntilFinish();
```

Imported from Jira [BEAM-3755](https://issues.apache.org/jira/browse/BEAM-3755). Original Jira may contain additional context.
Reported by: jbfbell.

Contributor guide

Open the contributing guide

Research direction

Start with the TestStream setup and the IntervalWindow and PAssert assertion shown in the issue; compare the event timestamps, watermark advances, and expected final pane. Reproduce the failing pipeline and determine whether the test or session-window merging behavior needs correction, with a passing assertion or documented expected result as the completion criterion.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
stream-processing, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.