apache / apache/hop

[Bug]: Unable to execute hive query

Open
#4,656 2 comments 0 reactions 0 assignees View on GitHub
awaiting triage Hop Server P2
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 32m
Merged PRs (30d)
216

Description

### Apache Hop version?

2.8.0

### Java version?

java11

### Operating system

Windows

### What happened?

We are encountering an issue while performing ElasticSearch (ES) synchronization with Hive using the Hop service. The Hive query executes successfully when run directly through Hive, but the same query fails when executed via the Hop service.

This behavior is blocking our workflow, and we need assistance to resolve it.

Exception in application logs
2024-11-29 04:42:21,034 WARN [main] org.apache.hadoop.mapred.YarnChild:187 Exception running child : java.io.IOException: Cannot create an instance of InputSplit class = org.elasticsearch.hadoop.hive.EsHiveInputFormat$EsHiveSplit:Class org.elasticsearch.hadoop.hive.EsHiveInputFormat$EsHiveSplit not found
at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:179)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:71)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:373)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:433)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:344)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
Caused by: java.lang.ClassNotFoundException: Class org.elasticsearch.hadoop.hive.EsHiveInputFormat$EsHiveSplit not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2332)
at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:175)
... 10 more

### Issue Priority

Priority: 0

### Issue Component

Component: Hop Server

Contributor guide

Open the contributing guide

Research direction

Start with the Hop Server path used for Elasticsearch synchronization with Hive and reproduce the query on Windows with Java 11, comparing direct Hive execution with the service logs. Done means identifying why EsHiveInputFormat$EsHiveSplit is unavailable in the service job and documenting or validating a fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.