linkedin / linkedin/dr-elephant
How and where to change the spark history server port number in dr-elephant
@ShubhamGupta29 is already working on this.
Since Apr 29, 2020.
- Dominant language
- Java
- Stars
- 1.4k
- Forks
- 839
- PR merge metrics
- No merged PRs in 30d
Description
Spark 2.4.4 version
emr release 5.29
Due to some constraint, i can not use port number 18080 for spark history server. Spark history server points to other than 18080(example: 18480).
I am getting below error message in dr-elephant log.
04-29-2020 05:31:54 ERROR [ForkJoinPool-1-worker-29] com.linkedin.drelephant.spark.fetchers.SparkRestClient : error reading applicationInfo http://master-dns-name:18080/api/v1/applications/application_1588136450759_0001. Exception Message = java.net.SocketTimeoutException: connect timed out
I have followed this guide to install dr.elephant in EMR- https://aws.amazon.com/blogs/big-data/tune-hadoop-and-spark-performance-with-dr-elephant-and-sparklens-on-amazon-emr/
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.