apache / apache/beam

[GSoC 2026] Kafka Streams runner: un-sickbay ParDoTest$LifecycleTests.testParDoWithErrorInStartBatch

Open
#39,452 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

## Summary

`ParDoTest$LifecycleTests.testParDoWithErrorInStartBatch` is sickbayed in the Kafka Streams runner's `validatesRunner` task (added in the ParDoTest sweep PR). This issue tracks un-sickbaying it.

### Why it's sickbayed

The test runs a DoFn whose `@StartBundle` throws, and asserts `run()` surfaces the user's error. Instead the run blocks for the SDK data service's three-minute timeout and then fails with a `TimeoutException` rather than the user's exception.

The DoFn fails during bundle setup, so the SDK harness never opens its data stream. Meanwhile `SdkHarnessClient.newBundle` has already sent the `ProcessBundleRequest` and then blocks in `GrpcDataService.createOutboundAggregator`, which waits up to three minutes (`connectedClient.get(3, TimeUnit.MINUTES)`) for that data stream. So the error on the control channel never gets a chance to surface — the run just waits out the timeout.

This is in shared `java-fn-execution` code, not runner-specific:
- The Flink runner sickbays all of `LifecycleTests` (`runners/flink/flink_runner.gradle`), with the note that its errors are non-deterministic and the real cause is only in the logs.
- The Prism runner sickbays all three of the `LifecycleTests` error tests (`runners/prism/java/build.gradle`).

The `@ProcessElement` and `@FinishBundle` variants pass in the Kafka Streams runner, because by the time those run the data stream is already established.

### Possible directions

- Detect a bundle that failed on the control channel and stop waiting on the outbound aggregator, so the control-channel error surfaces instead of the data-stream timeout.
- Or shorten the wait for this test setup so the run fails fast with the real cause.

Either likely belongs in `java-fn-execution` and would benefit the Flink and Prism runners too, so worth checking with those owners before changing shared code.

Part of #18479.

Contributor guide

Open the contributing guide

Research direction

Start with ParDoTest$LifecycleTests.testParDoWithErrorInStartBatch and read SdkHarnessClient.newBundle plus GrpcDataService.createOutboundAggregator in the shared java-fn-execution code. Compare the Kafka Streams behavior with the sickbay notes in runners/flink/flink_runner.gradle and runners/prism/java/build.gradle. Done means the test surfaces the @StartBundle error instead of waiting for the data-service timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, java, kafka
Domain
distributed-systems, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.