linkedin / linkedin/dr-elephant
Spark 2.3 is now out... Will Dr.elephant support it out of the box?
Open
- Dominant language
- Java
- Stars
- 1.4k
- Forks
- 839
- PR merge metrics
- No merged PRs in 30d
Description
https://spark.apache.org/releases/spark-release-2-3-0.html
So ive been reading about how linkedin has been using a modified SHS to pull from 1.X 2.X...
But if you were to install Spark 2.3, will DR.elephant just work with it out of the box?
Has anyone tried Spark 2.3 and DR elephant yet?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked Spark 2.3 release notes and review Dr. Elephant's Spark and SHS integration points. Determine whether Spark 2.3 works without changes; if not, identify the compatibility work and the tests needed to demonstrate support.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100