SparkRunnerDebugger fails when provided with a spark context in pipeline options
Open
bug
P3
runners
spark
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
SparkRunnerDebugger fails when provided with a spark context in pipeline options
Spark can have only one context running, trying to initialise a new context results in an exception.
Need to check if a context is provided
Imported from Jira [BEAM-1821](https://issues.apache.org/jira/browse/BEAM-1821). Original Jira may contain additional context.
Reported by: ksalant.
Contributor guide
Research direction
Start at the SparkRunnerDebugger entry point and trace how pipeline options are handled when a Spark context is supplied. Verify the existing context is detected instead of initializing another one, and confirm the single-context exception no longer occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100