Add SetUp() and TearDown() for CombineFns
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
I have a CombineFn that has a large amount of state that needs to be loaded once before it can add_input or merge_combiners (for example, the CombineFn might load up a large lookup table used for combining).
Right now, to initialise this state, for each of the methods, I check if the state has already been initialised, and if not, I initialise it. It would be nice if CombineFn provided a SetUp() method that is called once to initialise this state (and a corresponding TearDown() method to clean up this state if necessary).
Imported from Jira [BEAM-3736](https://issues.apache.org/jira/browse/BEAM-3736). Original Jira may contain additional context.
Reported by: chuanyu.
Contributor guide
Research direction
Start by locating CombineFn and its add_input and merge_combiners methods. Review how state is initialized and cleaned up today, then determine the lifecycle and runner semantics needed for SetUp() and TearDown(); done means the requested hooks are specified and covered by appropriate tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100