[Failing Test]: GroupByKeyTest$BasicTests$.testAfterProcessingTimeContinuationTriggerUsingState()
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What happened?
When the validation test for Flink runner runs, `GroupByKeyTest$BasicTests$.testAfterProcessingTimeContinuationTriggerUsingState()` fails from time to time. I checked the test but had some issue understanding the expected behavior.
As far as I understand, the expected simplified input and output sequence of the PTransforms are following:
1. Source
a. emit `0`
b. emit `Watermark.MAX_VALUE`
2. WithKeys
a. [in] `0`
b. [out] `KV{"dummy key", 0}`
c. [in] `Watermark.MAX_VALUE`
d. [out] `Watermark.MAX_VALUE`
3. ParDo
a. [in] `KV{"dummy key", 0}`
b. [out] `42`
c. [in] `Watermark.MAX_VALUE`
d. [out] `Watermark.MAX_VALUE` (which has a delay of 500 ms after it receives the `Watermark.MAX_VALUE`)
4. First Sum
a. [in] `42`
b. [out] `{42, Timing=EARLY}` (emitted 1 ms after it receives the input of `42`)
c. [in] `Watermark.MAX_VALUE` (roughly 500 ms after it emits `{42, Timing=EARLY}`
d. [out] `{42, Timing=ON_TIME}`
e. [out] `Watermark.MAX_VALUE`
5. Second Sum
a. [in] `{42, Timing=EARLY}`
b. [out] `(42, Timing=EARLY)` (emitted 1 ms after it receives the input of `{42, Timing=EARLY}`
c. [in] `{42, Timing=ON_TIME}`
d. [out] `{84, Timing=EARLY}` (emitted 1 ms after it receives the input of `{42, Timing=ON_TIME}`, if the `Watermark.MAX_VALUE` hasn't arrived by then. Otherwise, this output will be skipped.)
e. [in] `Watermark.MAX_VALUE`
f. [out] `{84, Timing=ON_TIME}`
I am wondering if the race condition between 5.d and 5.e causes the test to be flaky.
- When 5.e happens before 5.d, the test will pass because there will only be one early fire which is 42.
- When 5.e happens after 5.d, the test will fail because both 42 and 84 are showing up as early fires.
However, I tried to tweak the delay of `AfterProcessingTime.pastFirstElementInPane()` a little bit, the test seems still failing ~50% of the runs. So I am not quite sure if my theory above makes sense.
@kennknowles It looks like this test was added by you. Could you help check? Thanks.
### Issue Failure
Failure: Test is flaky
### Issue Priority
Priority: 2 (backlog / disabled test but we think the product is healthy)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [X] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
Contributor guide
Assessment
This issue has not been assessed yet.