[Task]: Improve documentation of PortableRunner for usage with Spark
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What needs to happen?
Users are struggling to use the portable runner with Spark. Related Spark documentation should be updated and provide detailed instructions.
> While users will probably still prefer the classic Spark runner for most Java pipelines, the Spark portable runner enables cross-language transforms. We can start by providing instructions for plain Java, then add cross-language (Java -> Python).
See:
- https://stackoverflow.com/questions/66320831/its-possible-to-configure-the-beam-portable-runner-with-the-spark-configuration
- https://stackoverflow.com/questions/66498209/how-to-configure-beam-python-sdk-with-spark-in-a-kubernetes-environment
Also related: https://github.com/apache/beam/issues/21092
### Issue Priority
Priority: 2
### Issue Component
Component: runner-spark
Contributor guide
Assessment
This issue has not been assessed yet.