DTStack / DTStack/chunjun

[Question][Hive3Sink] kafka data to hive3 table :.ClassNotFoundException: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat

Open
#1,228 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
4.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Search before asking

- [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar question.

- [ ] I had googled my question but i didn't get any help.

- [X] I had read the documentation: [ChunJun doc](https://dtstack.github.io/chunjun) but it didn't help me.

### Description

我在使用chunjun 1.12.5采集Kafka数据入Hive3 orc表的时候,报如下错误:java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat

我的环境信息:
Hadoop:3.1.4
Chunjun:1.12.5
Kafka:2.1
Hive:3.1.4

**详细日志**:
INFO [main] org.apache.flink.configuration.GlobalConfiguration:Loading configuration property: fs.hdfs.hadoopconf, /soft/hadoop-3.1.4/etc/hadoop
INFO [main] com.dtstack.chunjun.Main:Register to table configuration:{table.dynamic-table-options.enabled=true, pipeline.name=Flink_Job}
INFO [main] com.dtstack.chunjun.util.PluginUtil:ChunJun executionMode: local
INFO [main] com.dtstack.chunjun.util.PluginUtil:ChunJun reset pipeline.jars: [/soft/chunjun-dist-v1.12.5/chunjun-dist/chunjun-core.jar, /soft/chunjun-dist-v1.12.5/chunjun-dist/metrics/prometheus/chunjun-metrics-prometheus.jar, /soft/chunjun-dist-v1.12.5/chunjun-dist/dirty-data-collector/log/chunjun-dirty-log.jar, /soft/chunjun-dist-v1.12.5/chunjun-dist/connector/hive3/chunjun-connector-hive3.jar, /soft/chunjun-dist-v1.12.5/chunjun-dist/connector/kafka/chunjun-connector-kafka.jar]
INFO [main] com.dtstack.chunjun.classloader.ClassLoaderManager:jarUrl:file:/soft/chunjun-dist-v1.12.5/chunjun-dist/chunjun-core.jar_file:/soft/chunjun-dist-v1.12.5/chunjun-dist/connector/hive3/chunjun-connector-hive3.jar_file:/soft/chunjun-dist-v1.12.5/chunjun-dist/connector/kafka/chunjun-connector-kafka.jar_file:/soft/chunjun-dist-v1.12.5/chunjun-dist/dirty-data-collector/log/chunjun-dirty-log.jar_file:/soft/chunjun-dist-v1.12.5/chunjun-dist/metrics/prometheus/chunjun-metrics-prometheus.jar create ClassLoad successful...
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/io/orc/OrcOutputFormat
at com.dtstack.chunjun.connector.hive3.sink.Hive3OutputFormatBuilder.newBuild(Hive3OutputFormatBuilder.java:37)
at com.dtstack.chunjun.connector.hive3.sink.Hive3SinkFactory.createSink(Hive3SinkFactory.java:67)
at com.dtstack.chunjun.Main.exeSyncJob(Main.java:222)
at com.dtstack.chunjun.Main.main(Main.java:122)
at com.dtstack.chunjun.client.local.LocalClusterClientHelper.submit(LocalClusterClientHelper.java:35)
at com.dtstack.chunjun.client.Launcher.main(Launcher.java:119)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 6 more

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.