The DirectRunner can complete before reporting the results of all ModelEnforcements
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
The termination condition in the ExecutorServiceParallelExecutor.MonitorRunnable checks to see if the evaluation context is done, among other things. Due to the signatures of ModelEnforcement, output is committed before ModelEnforcement runs to ensure that the enforcement has access to the final elements. If the transform that performs ModelEnforcement completes, and all PTransforms are done, the pipeline can shutdown before model enforcement fails.
Imported from Jira [BEAM-819](https://issues.apache.org/jira/browse/BEAM-819). Original Jira may contain additional context.
Reported by: tgroh.
Contributor guide
Research direction
Start with DirectRunner's ExecutorServiceParallelExecutor.MonitorRunnable and trace how its termination condition observes the evaluation context and completed PTransforms. Review the ModelEnforcement execution path, then verify that the DirectRunner remains alive until all ModelEnforcements have reported their results, including failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100