Fix context classloader in Spark portable runner
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Spark is setting the context class loader to support dynamic class loading, leading to unpredictable behavior with duplicate jars being found on the class path. We need to see if there is a way to disable this behavior so we can use the context class loader deterministically.
Imported from Jira [BEAM-10109](https://issues.apache.org/jira/browse/BEAM-10109). Original Jira may contain additional context.
Reported by: ibzib.
Contributor guide
Research direction
Start by reading the Spark portable runner implementation and the linked BEAM-10109 Jira issue for the missing context. Determine how Spark sets the context class loader and whether that behavior can be disabled; done means duplicate jars no longer cause nondeterministic class loading.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100