linkedin / linkedin/dr-elephant
Spark version compatibility and compilation error for Spark 2.0.1
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.4k
- Forks
- 839
- PR merge metrics
- No merged PRs in 30d
Description
Hi all,
I'm trying compile latest master branch with spark 2.0.1 and hadoop 2.7.3 and I'm getting the following error:
```
[error] impossible to get artifacts when data has not been loaded. IvyNode = com.fasterxml.jackson.core#jackson-databind;2.5.4
java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = com.fasterxml.jackson.core#jackson-databind;2.5.4
```
Setting hadoop 2.4.0 and spark 1.6.1. compiles fine though. Could someone point me to documentation stating which spark and hadoop versions are supported? Should I manually set the dependencies and if so how?
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file or test is named. Reproduce the latest master build with Spark 2.0.1 and Hadoop 2.7.3, compare it with the working Spark 1.6.1 and Hadoop 2.4.0 combination, and inspect the build dependency configuration. Done means the supported Spark and Hadoop versions are documented and the reported compilation failure or dependency guidance is resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java, spark
- Domain
- build-system, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100