apache / apache/incubator-xtable

Missing org.openjdk.jol from imported dependencies in xtable-utilities makes it to fail with NoClassDefFoundError

Open Beginner friendly
#736 8 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
1.2k
Forks
212
Avg merge
4d 9h
Merged PRs (30d)
16

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/incubator-xtable/issues?q=is%3Aissue) and found no similar issues.

### Please describe the bug 🐞

I cloned the repo then moved to `0.3.0-incubating`:

```
git checkout 0.3.0-incubating
```

then moved to the `xtable-utilities` folder

```
cd xtable-utilities
```

and compiled it

```
mvn clean package -DskipTests
```

Then run the tool with proper conf files

```
java -jar xtable-utilities/target/xtable-utilities_2.12-0.3.0-incubating-bundled.jar --datasetConfig datasetConfig.yaml --icebergCatalogConfig glueDataCatalog.yaml
```

it failed with

> Exception in thread "main" java.lang.NoClassDefFoundError: org/openjdk/jol/info/GraphLayout at org.apache.hudi.common.util.ObjectSizeCalculator.getObjectSize(ObjectSizeCalculator.java:57) at org.apache.hudi.common.util.HoodieRecordSizeEstimator.(HoodieRecordSizeEstimator.java:40) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.(HoodieMergedLogRecordScanner.java:109) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.(HoodieMergedLogRecordScanner.java:76) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner$Builder.build(HoodieMergedLogRecordScanner.java:466) at org.apache.hudi.metadata.HoodieMetadataLogRecordReader$Builder.build(HoodieMetadataLogRecordReader.java:219) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getLogRecordScanner(HoodieBackedTableMetadata.java:498) at org.apache.hudi.metadata.HoodieBackedTableMetadata.openReaders(HoodieBackedTableMetadata.java:429) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getOrCreateReaders$10(HoodieBackedTableMetadata.java:412) at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getOrCreateReaders(HoodieBackedTableMetadata.java:412) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lookupKeysFromFileSlice(HoodieBackedTableMetadata.java:291) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordsByKeys(HoodieBackedTableMetadata.java:255) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordByKey(HoodieBackedTableMetadata.java:145) at org.apache.hudi.metadata.BaseTableMetadata.fetchAllPartitionPaths(BaseTableMetadata.java:316) at org.apache.hudi.metadata.BaseTableMetadata.getAllPartitionPaths(BaseTableMetadata.java:125) at org.apache.xtable.hudi.HudiDataFileExtractor.getFilesCurrentState(HudiDataFileExtractor.java:116) at org.apache.xtable.hudi.HudiConversionSource.getCurrentSnapshot(HudiConversionSource.java:110) at org.apache.xtable.spi.extractor.ExtractFromSource.extractSnapshot(ExtractFromSource.java:40) at org.apache.xtable.conversion.ConversionController.syncSnapshot(ConversionController.java:281) at org.apache.xtable.conversion.ConversionController.syncTableFormats(ConversionController.java:203) at org.apache.xtable.conversion.ConversionController.sync(ConversionController.java:106) at org.apache.xtable.utilities.RunSync.syncTableMetdata(RunSync.java:186) at org.apache.xtable.utilities.RunSync.runSync(RunSync.java:297) at org.apache.xtable.utilities.RunSync.main(RunSync.java:282) Caused by: java.lang.ClassNotFoundException: org.openjdk.jol.info.GraphLayout at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525) ... 25 more

To fix I modified `xtable-utilities/pom.xml` and changed this:

```

org.openjdk.jol
jol-core
test

```
to:

```

org.openjdk.jol
jol-core
0.16

```

then recompiled with:

```
mvn clean package -DskipTests
```

this time running the tool everything is ok.

### Are you willing to submit PR?

- [ ] I am willing to submit a PR!
- [ ] I am willing to submit a PR but need help getting started!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in xtable-utilities/pom.xml and inspect how the jol-core dependency is scoped and included in the bundled artifact. Run mvn clean package -DskipTests, then launch the generated xtable-utilities_2.12-0.3.0-incubating-bundled.jar with the reported configuration files; done means the tool starts without NoClassDefFoundError for org.openjdk.jol.info.GraphLayout.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.