apache / apache/hudi

[SUPPORT] The Hive run_sync_tool's Logged Command & The Actual Command Do Not Match

Open
#11,029 2 comments 0 reactions 0 assignees View on GitHub
component:catalog-sync docs
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

This is a pretty small/minor issue I noticed while working with the `HiveSyncTool`. Essentially what's being logged does **not** match what's actually being executed:

https://github.com/apache/hudi/blob/40d4f489389083e3c6d69954361d3de4aec8186a/hudi-sync/hudi-hive-sync/run_sync_tool.sh#L54-L55

As you see in the above snippet, the JARs passed to the `java -cp` command have one order when they're logged and another when the command is actually executed. I made the below change on my end and was able to run the `HiveSyncTool` without any issues. But with the original ordering, I was getting `ClassNotFound` exception.

```diff
echo "Running Command : java -cp ${HADOOP_HIVE_JARS}:${HADOOP_CONF_DIR}:$HUDI_HIVE_UBER_JAR org.apache.hudi.hive.HiveSyncTool $@"
- java -cp $HUDI_HIVE_UBER_JAR:${HADOOP_HIVE_JARS}:${HADOOP_CONF_DIR} org.apache.hudi.hive.HiveSyncTool "$@"
+ java -cp ${HADOOP_HIVE_JARS}:${HADOOP_CONF_DIR}:$HUDI_HIVE_UBER_JAR org.apache.hudi.hive.HiveSyncTool "$@"
```

I'm happy to submit a PR IF you all think this is indeed something that should be resolved. Thanks!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with hudi-sync/hudi-hive-sync/run_sync_tool.sh at lines 54-55, comparing the logged java -cp command with the command that executes HiveSyncTool. Verify the classpath ordering and run the Hive sync tool or its available checks; done means the logged and executed commands match and the ClassNotFound failure is addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, shell
Domain
data-engineering
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.