Tear down unused DoFns periodically in Java SDK harness
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Per the discussion in the ML the detail can be found here[1], the teardown of DoFns should be supported in the portability framework. It happens at two places:
1) Upon the control service termination
2) Tear down the unused DoFns periodically
The aim of this JIRA is to add support for tear down the unused DoFns periodically in Java SDK harness.
[1] https://lists.apache.org/thread.html/0c4a4cf83cf2e35c3dfeb9d906e26cd82d3820968ba6f862f91739e4@%3Cdev.beam.apache.org%3E
Imported from Jira [BEAM-8620](https://issues.apache.org/jira/browse/BEAM-8620). Original Jira may contain additional context.
Reported by: sunjincheng121.
Contributor guide
Research direction
Start with the Java SDK harness and the portability framework discussion linked in the issue. Determine how unused DoFns are identified and where periodic teardown belongs, then verify that unused DoFns are torn down without affecting active processing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100