apache / apache/incubator-xtable
Missing org.openjdk.jol from imported dependencies in xtable-utilities makes it to fail with NoClassDefFoundError
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 212
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 16
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/incubator-xtable/issues?q=is%3Aissue) and found no similar issues.
### Please describe the bug 🐞
I cloned the repo then moved to `0.3.0-incubating`:
```
git checkout 0.3.0-incubating
```
then moved to the `xtable-utilities` folder
```
cd xtable-utilities
```
and compiled it
```
mvn clean package -DskipTests
```
Then run the tool with proper conf files
```
java -jar xtable-utilities/target/xtable-utilities_2.12-0.3.0-incubating-bundled.jar --datasetConfig datasetConfig.yaml --icebergCatalogConfig glueDataCatalog.yaml
```
it failed with
> Exception in thread "main" java.lang.NoClassDefFoundError: org/openjdk/jol/info/GraphLayout at org.apache.hudi.common.util.ObjectSizeCalculator.getObjectSize(ObjectSizeCalculator.java:57) at org.apache.hudi.common.util.HoodieRecordSizeEstimator.(HoodieRecordSizeEstimator.java:40) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.(HoodieMergedLogRecordScanner.java:109) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.(HoodieMergedLogRecordScanner.java:76) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner$Builder.build(HoodieMergedLogRecordScanner.java:466) at org.apache.hudi.metadata.HoodieMetadataLogRecordReader$Builder.build(HoodieMetadataLogRecordReader.java:219) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getLogRecordScanner(HoodieBackedTableMetadata.java:498) at org.apache.hudi.metadata.HoodieBackedTableMetadata.openReaders(HoodieBackedTableMetadata.java:429) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getOrCreateReaders$10(HoodieBackedTableMetadata.java:412) at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getOrCreateReaders(HoodieBackedTableMetadata.java:412) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lookupKeysFromFileSlice(HoodieBackedTableMetadata.java:291) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordsByKeys(HoodieBackedTableMetadata.java:255) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordByKey(HoodieBackedTableMetadata.java:145) at org.apache.hudi.metadata.BaseTableMetadata.fetchAllPartitionPaths(BaseTableMetadata.java:316) at org.apache.hudi.metadata.BaseTableMetadata.getAllPartitionPaths(BaseTableMetadata.java:125) at org.apache.xtable.hudi.HudiDataFileExtractor.getFilesCurrentState(HudiDataFileExtractor.java:116) at org.apache.xtable.hudi.HudiConversionSource.getCurrentSnapshot(HudiConversionSource.java:110) at org.apache.xtable.spi.extractor.ExtractFromSource.extractSnapshot(ExtractFromSource.java:40) at org.apache.xtable.conversion.ConversionController.syncSnapshot(ConversionController.java:281) at org.apache.xtable.conversion.ConversionController.syncTableFormats(ConversionController.java:203) at org.apache.xtable.conversion.ConversionController.sync(ConversionController.java:106) at org.apache.xtable.utilities.RunSync.syncTableMetdata(RunSync.java:186) at org.apache.xtable.utilities.RunSync.runSync(RunSync.java:297) at org.apache.xtable.utilities.RunSync.main(RunSync.java:282) Caused by: java.lang.ClassNotFoundException: org.openjdk.jol.info.GraphLayout at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525) ... 25 more
To fix I modified `xtable-utilities/pom.xml` and changed this:
```
org.openjdk.jol
jol-core
test
```
to:
```
org.openjdk.jol
jol-core
0.16
```
then recompiled with:
```
mvn clean package -DskipTests
```
this time running the tool everything is ok.
### Are you willing to submit PR?
- [ ] I am willing to submit a PR!
- [ ] I am willing to submit a PR but need help getting started!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in xtable-utilities/pom.xml and inspect how the jol-core dependency is scoped and included in the bundled artifact. Run mvn clean package -DskipTests, then launch the generated xtable-utilities_2.12-0.3.0-incubating-bundled.jar with the reported configuration files; done means the tool starts without NoClassDefFoundError for org.openjdk.jol.info.GraphLayout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100