How to use Kerberos in Hudi integrated Flink?data cannot be written to hdfs.
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
I have configured the Kerberos-related parameters in the Flink configuration file and executed the kinit command before entering SQL client, but it still failed.
**Environment Description**
* Hudi version : 0.14.0
* Flink version : 1.16.2
* Hadoop version :3.1.0
* Storage (HDFS/S3/GCS..) : hdfs
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the attached screenshots and the Kerberos-related Flink configuration described in the issue, then reproduce the failure from the Flink SQL client after running kinit. Check whether the configured HDFS authentication permits the Hudi write; done means data can be written to HDFS with Hudi 0.14.0, Flink 1.16.2, and Hadoop 3.1.0.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- authentication, data-engineering, distributed-systems, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100