apache / apache/kyuubi

[Improvement] Any plans for Kyuubi Flink Engine on YARN to support Kerberos?

Open
#6,850 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
2.4k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.

### What would you like to be improved?

I have flink, hive hadoop cluster with kerberos, and want to use kyuubi access flink sql using yarn-session mode
so i get some error:
[kyuubi-flink-sql-engine.log](https://github.com/user-attachments/files/18128900/kyuubi-flink-sql-engine.log)
[kyuubi-server-hadoop98.test.com.log](https://github.com/user-attachments/files/18128902/kyuubi-server-hadoop98.test.com.log)

I found this guide: https://kyuubi.readthedocs.io/en/master/deployment/engine_on_yarn.html#deploy-kyuubi-flink-engine-on-yarn
> Deploy Kyuubi Flink Engine on YARN,YARN Session Mode,Kerberos
> With regard to YARN session mode, security.kerberos.login.keytab and security.kerberos.login.principal are not effective, as Kyuubi Flink SQL engine mainly relies on Flink SQL client which currently does not support

so, Are there any plans for Kyuubi Flink Engine on YARN to support Kerberos?

### How should we improve?

I don't know.

Maybe:
kyuubi/kyuubi-server/src/main/scala/org/apache/kyuubi/client/KyuubiSyncThriftClient.scala add some method like getPlainTransport() and support SASL. i am not sure

### Are you willing to submit PR?

- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [X] No. I cannot submit a PR at this time.

Contributor guide

Open the contributing guide

Research direction

Start with the deployment guide section for Kyuubi Flink Engine on YARN in YARN session mode and review the attached engine and server logs. Then inspect kyuubi-server/src/main/scala/org/apache/kyuubi/client/KyuubiSyncThriftClient.scala and the surrounding Flink-on-YARN entry points. Done requires an agreed, documented and tested approach for Kerberos support in this mode.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, scala
Domain
backend, distributed-systems, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.