Spark uber jar job server: query exceptions from master
Open
improvement
P3
portability
runners
spark
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
As far as I know, the Spark REST API does not return exceptions from the cluster after a jar is actually run. While these exceptions can be viewed in Spark's web UI, ideally they would also be visible in Beam's output.
To do this, we will need to find a REST endpoint that does return those exceptions, and then map the submissionId to its corresponding job id.
Imported from Jira [BEAM-8983](https://issues.apache.org/jira/browse/BEAM-8983). Original Jira may contain additional context.
Reported by: ibzib.
Contributor guide
Assessment
This issue has not been assessed yet.