[GSoC 2026] Kafka Streams runner: un-sickbay ParDoTest$LifecycleTests.testParDoWithErrorInStartBatch
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
## Summary
`ParDoTest$LifecycleTests.testParDoWithErrorInStartBatch` is sickbayed in the Kafka Streams runner's `validatesRunner` task (added in the ParDoTest sweep PR). This issue tracks un-sickbaying it.
### Why it's sickbayed
The test runs a DoFn whose `@StartBundle` throws, and asserts `run()` surfaces the user's error. Instead the run blocks for the SDK data service's three-minute timeout and then fails with a `TimeoutException` rather than the user's exception.
The DoFn fails during bundle setup, so the SDK harness never opens its data stream. Meanwhile `SdkHarnessClient.newBundle` has already sent the `ProcessBundleRequest` and then blocks in `GrpcDataService.createOutboundAggregator`, which waits up to three minutes (`connectedClient.get(3, TimeUnit.MINUTES)`) for that data stream. So the error on the control channel never gets a chance to surface — the run just waits out the timeout.
This is in shared `java-fn-execution` code, not runner-specific:
- The Flink runner sickbays all of `LifecycleTests` (`runners/flink/flink_runner.gradle`), with the note that its errors are non-deterministic and the real cause is only in the logs.
- The Prism runner sickbays all three of the `LifecycleTests` error tests (`runners/prism/java/build.gradle`).
The `@ProcessElement` and `@FinishBundle` variants pass in the Kafka Streams runner, because by the time those run the data stream is already established.
### Possible directions
- Detect a bundle that failed on the control channel and stop waiting on the outbound aggregator, so the control-channel error surfaces instead of the data-stream timeout.
- Or shorten the wait for this test setup so the run fails fast with the real cause.
Either likely belongs in `java-fn-execution` and would benefit the Flink and Prism runners too, so worth checking with those owners before changing shared code.
Part of #18479.
Contributor guide
Research direction
Start with ParDoTest$LifecycleTests.testParDoWithErrorInStartBatch and read SdkHarnessClient.newBundle plus GrpcDataService.createOutboundAggregator in the shared java-fn-execution code. Compare the Kafka Streams behavior with the sickbay notes in runners/flink/flink_runner.gradle and runners/prism/java/build.gradle. Done means the test surfaces the @StartBundle error instead of waiting for the data-service timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grpc, java, kafka
- Domain
- distributed-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100