linkedin / linkedin/dr-elephant
Dr Elephant dashboard not showing all the Jobs
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.4k
- Forks
- 839
- PR merge metrics
- No merged PRs in 30d
Description
I have been able to deploy Dr. Elephant but it is only picking 1 hive job.
The cluster has 3 master nodes. Dr. E has been deployed on master node 3, where that job was ran.

Note: I have installed MySQL in master node 3 as well.
When I check the logs on dr_elephant.log I seeother jobs from master node 2, but they don't show on the dashboard. And I am also getting the following errors:

It says that I have to configure SPARK_HOME and SPARK_CONF_DIR,
But I already did in **.bash_profile**:
PATH=$PATH:$HOME/bin
export PATH
export PATH=$PATH:/usr/jdk64/jdk1.8.0_112/bin
export JAVA_HOME=/usr/jdk64/jdk1.8.0_112
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
export HADOOP_HOME=/usr/hdp/current/hadoop-client
export HADOOP_CONF_DIR=/usr/hdp/current/hadoop-client/conf
export SPARK_HOME=/usr/hdp/current/spark-client
export SPARK_CONF_DIR=/usr/hdp/current/spark-client/conf
export PATH=$HADOOP_HOME/bin:$PATH
export PATH=$PATH:/root/activator-dist-1.3.12/bin/
I also get the following errors:

and

Thanks for the help
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Dr. Elephant logs and the deployment configuration on the three master nodes, comparing the environment shown in .bash_profile with the process environment. Investigate why jobs reported from master node 2 do not appear in the dashboard and review the attached SPARK_HOME, SPARK_CONF_DIR, and other error messages. Done means jobs from the cluster are consistently visible in the dashboard without the reported configuration errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java, mysql
- Domain
- databases, distributed-systems, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100