linkedin / linkedin/dr-elephant
Dr Elephant requires optional hadoop configuration parameters to find YARN RM, JobHistory
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.4k
- Forks
- 839
- PR merge metrics
- No merged PRs in 30d
Description
The Dr Elephant fetcher code requires the `yarn.resourcemanager.webapp.address` parameter, which is an optional configuration for Apache Hadoop. The parameter `yarn.resourcemanager.hostname` will automatically populate all `yarn.resourcemanager.*.address` parameters with default ports. dr-elephant should support this.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Dr Elephant fetcher code that locates YARN ResourceManager and JobHistory using Hadoop configuration. Check how yarn.resourcemanager.webapp.address and yarn.resourcemanager.hostname are resolved, then verify that hostname-based default addresses work without the optional webapp address; add or update the relevant fetcher tests if present.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100