apache / apache/kyuubi

[FEATURE] Support renew delegation tokens of multiple Hive metastore clusters

Open
#1,091 1 comment 1 reaction 0 assignees View on GitHub
kind:feature
Dominant language
Scala
Stars
2.4k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.

### Describe the feature

Enhance `HiveDelegationTokenProvider` to renew delegation tokens of multiple Hive metastore clusters.

### Motivation

Kyuubi supports [setting Hive metastore uris in JDBC connection URL](https://kyuubi.readthedocs.io/en/latest/deployment/hive_metastore.html#via-jdbc-connection-url).
But current `HiveDelegationTokenProvider` implemented in #1009 can only renew delegation tokens of Hive metastore defined in hive-site.xml.

For users who frequently access different Hive metastore clusters, the Kyuubi enginesare still limited by Hive token's expiration time when visiting non default Hive metastore servers.

### Describe the solution

Introduce a Kyuubi configuration `kyuuib.credentials.hive.metastore.uris` to specify extra Hive metastore URIs.
URIs should be specified in format "thrift://host01:port01,thrift://host02:port02;thrift://host11:port11,thrift://host12:port12", where "," separates URIs belonging to same metastore cluster, ";" separates URIs of different metastore clusters.

`HiveDelegationTokenProvider` will create a `HiveMetaStoreClient` for each Hive metastore URIs and periodically renew the delegation tokens.

### Additional context

_No response_

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the HiveDelegationTokenProvider implementation and trace its current Hive metastore URI configuration and delegation-token renewal flow. Review how the proposed kyuuib.credentials.hive.metastore.uris format should distinguish URIs within and across clusters; done means tokens for every configured metastore cluster are periodically renewed.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.