apache / apache/kyuubi

[Bug] kyuubi-1.7 can't use mutil metastores

Open
#5,181 3 comments 0 reactions 0 assignees View on GitHub
kind:bug priority:major
Dominant language
Scala
Stars
2.4k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.

### Describe the bug

the case is after configuring iceberg metastore which is different from hive metastore(means i have two different hive metastores), when connecting iceberg catalog. metastore can't be connected cause delegation token expire errors.

```
Caused by: org.apache.hadoop.security.token.SecretManager$InvalidToken: token expired or does not exist: HIVE_DELEGATION_TOKEN owner=hive, renewer=hive, realUser=hive/hostxxxxxxxxxxxx, issueDate=1692543549564, maxDate=1693148349564, sequenceNumber=67, masterKeyId=1
at org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.retrievePassword(TokenStoreDelegationTokenSecretManager.java:114)
at org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.retrievePassword(TokenStoreDelegationTokenSecretManager.java:56)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$SaslDigestCallbackHandler.getPassword(HadoopThriftAuthBridge.java:565)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$SaslDigestCallbackHandler.handle(HadoopThriftAuthBridge.java:596)
at com.sun.security.sasl.digest.DigestMD5Server.validateClientResponse(DigestMD5Server.java:589)
```

why i said this is bug, since when i use original spark-sql command, it works fine.

here examples:
```
spark-sql (default)>
> use hive_prod;
spark-sql (default)>
>
> show databases;
default
Time taken: 0.689 seconds, Fetched 1 row(s)
spark-sql (default)>
```
but kyuubi-1.7 failed.

### Affects Version(s)

1.7

### Kyuubi Server Log Output

_No response_

### Kyuubi Engine Log Output

_No response_

### Kyuubi Server Configurations

_No response_

### Kyuubi Engine Configurations

_No response_

### Additional context

_No response_

### Are you willing to submit PR?

- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [x] No. I cannot submit a PR at this time.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Kyuubi 1.7 connection to an Iceberg catalog using a Hive metastore distinct from the default metastore, then compare it with the working spark-sql case. Use the reported InvalidToken trace to investigate delegation-token handling. Done means the alternate metastore connects without token-expiration failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, scala, sql
Domain
databases, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.