Error: Failed to load class com.linkedin.dualip.preprocess.MatchingDataGenerator.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 87
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
Try to follow the instructions in the document. When I try to run the demo code to covert the matching data, the error pops out. Here is the full log:
:: loading settings :: url = jar:file:/usr/local/spark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
Ivy Default Cache set to: /Users/liubo.li/.ivy2/cache
The jars for the packages stored in: /Users/liubo.li/.ivy2/jars
org.apache.spark#spark-avro_2.12 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-14453ece-ca22-46fe-966c-cc48203274b1;1.0
confs: [default]
found org.apache.spark#spark-avro_2.12;3.1.1 in central
found org.spark-project.spark#unused;1.0.0 in central
:: resolution report :: resolve 217ms :: artifacts dl 4ms
:: modules in use:
org.apache.spark#spark-avro_2.12;3.1.1 from central in [default]
org.spark-project.spark#unused;1.0.0 from central in [default]
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 2 | 0 | 0 | 0 || 2 | 0 |
---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent-14453ece-ca22-46fe-966c-cc48203274b1
confs: [default]
0 artifacts copied, 2 already retrieved (0kB/4ms)
23/10/17 12:01:41 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Error: Failed to load class com.linkedin.dualip.preprocess.MatchingDataGenerator.
log4j:WARN No appenders could be found for logger (org.apache.spark.util.ShutdownHookManager).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by retracing the document's demo instructions and the command used to convert the matching data, then investigate why Spark cannot load com.linkedin.dualip.preprocess.MatchingDataGenerator. The work is done when the documented demo runs successfully and converts the matching data without the class-loading error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100