[Bug]: Unable to execute hive query
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 18h 32m
- Merged PRs (30d)
- 216
Description
### Apache Hop version?
2.8.0
### Java version?
java11
### Operating system
Windows
### What happened?
We are encountering an issue while performing ElasticSearch (ES) synchronization with Hive using the Hop service. The Hive query executes successfully when run directly through Hive, but the same query fails when executed via the Hop service.
This behavior is blocking our workflow, and we need assistance to resolve it.
Exception in application logs
2024-11-29 04:42:21,034 WARN [main] org.apache.hadoop.mapred.YarnChild:187 Exception running child : java.io.IOException: Cannot create an instance of InputSplit class = org.elasticsearch.hadoop.hive.EsHiveInputFormat$EsHiveSplit:Class org.elasticsearch.hadoop.hive.EsHiveInputFormat$EsHiveSplit not found
at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:179)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:71)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:373)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:433)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:344)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
Caused by: java.lang.ClassNotFoundException: Class org.elasticsearch.hadoop.hive.EsHiveInputFormat$EsHiveSplit not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2332)
at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:175)
... 10 more
### Issue Priority
Priority: 0
### Issue Component
Component: Hop Server
Contributor guide
Research direction
Start with the Hop Server path used for Elasticsearch synchronization with Hive and reproduce the query on Windows with Java 11, comparing direct Hive execution with the service logs. Done means identifying why EsHiveInputFormat$EsHiveSplit is unavailable in the service job and documenting or validating a fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100