Python SparkRunner should connect to secure cluster
Open
improvement
P3
portability
runners
spark
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Currently, there is no way for the Python SparkRunner to connect to a secure Spark cluster. There should be.
Imported from Jira [BEAM-10877](https://issues.apache.org/jira/browse/BEAM-10877). Original Jira may contain additional context.
Reported by: ibzib.
Contributor guide
Research direction
Start with the Python SparkRunner entry point and the imported Jira issue BEAM-10877, since no repository file or test is named. Determine the secure Spark cluster connection requirements and define validation for a successful connection; done means Python SparkRunner can connect to a secure Spark cluster.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, spark
- Domain
- data-engineering, distributed-systems, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100