apache / apache/hudi

[SUPPORT] Hudi HMS Catalog hive_sync.conf.dir

Open
#9,269 6 comments 0 reactions 1 assignee Claimed by @danny0405 View on GitHub
area:config priority:high
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

**_Tips before filing an issue_**

- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?

- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.

- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.

**Describe the problem you faced**

flink: 1.15, flink on yarn, submit jobgraph trough java client
hudi: 0.13.1 compile ourself with tags/0.13.1 and more logs for debug
hive: 3.1.0, kerberized

we use hudi hms catalog, and config `hive_sync.conf.dir` as local hive-site.xml folder, but it seems have not read the local hive-site.xml file

**To Reproduce**

Steps to reproduce the behavior:

1. compile tags/0.13.1 with more logs and no other changes
2. config flink sql with hive_sync.conf.dir , set to local hive conf dir include a hive-site.xml file
3. parse flink sql to job graph and submit job graph through java client to yarn cluster as per-job mode
4. find wrong hiveConf values in HiveSyncContext `e.g. hive.metastore.kerberos.principal: hive-metastore/_HOST@EXAMPLE.COM` like same kind of default value

**Expected behavior**

hive_sync works as expected

should the `hive_sync.conf.dir` set to an hdfs path, since the code running on yarn and my hive-site.xml is in a local dir?

**Environment Description**

* Hudi version : 0.13.1

* Hive version : 3.1.0 kerberized

* Hadoop version : 3.1.1

* Storage (HDFS/S3/GCS..) : HDFS

* Running on Docker? (yes/no) : no

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.