apache / apache/kyuubi

[Bug] kyuubi combines the spark plugin built-in in emr

Open
#6,198 9 comments 0 reactions 0 assignees View on GitHub
kind:bug priority:major
Dominant language
Scala
Stars
2.4k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.

### Describe the bug

Approved by kinit and connected via beeline, but kept reporting an error that the hadoop user had no permissions, users who appear to have directly used the kyuubi startup service
```
[huchao@ip-10-0-20-117 hadoop]$ klist
Ticket cache: FILE:/tmp/krb5cc_2702
Default principal: huchao@COMPUTE.INTERNAL

Valid starting Expires Service principal
03/19/2024 01:57:02 03/19/2024 11:57:02 krbtgt/COMPUTE.INTERNAL@COMPUTE.INTERNAL
renew until 03/20/2024 01:57:02
[hadoop@ip-10-0-20-117 ~]$ sudo ps -ef|grep kyuubi
hadoop 11150 29130 0 05:20 pts/3 00:00:00 grep --color=auto kyuubi
hadoop 26559 1 3 05:14 pts/1 00:00:15 /etc/alternatives/jre/bin/java -XX:+IgnoreUnrecognizedVMOptions -Dio.netty.tryReflectionSetAccessible=true --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.security.tools.keytool=ALL-UNNAMED --add-opens=java.base/sun.security.x509=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED -cp /mnt/kyuubi/conf:/mnt/kyuubi/web-ui:/mnt/kyuubi/jars/*:/etc/hadoop/conf org.apache.kyuubi.server.KyuubiServer
```
### Affects Version(s)

1.8.1

### Kyuubi Server Log Output

```logtalk
Connection command:
beeline -u "jdbc:hive2://$(hostname -f):10009/default; principal=hive/$(hostname -f)@COMPUTE.INTERNAL"
But an error is reported: Caused by: org.apache.spark.sql.catalyst.analysis.AccessControlException:
com.amazonaws.emr.recordserver.remote.AuthorizationException: [ Permission Denied: User hadoop does not have _any permission on default ]
at com.amazonaws.emr.recordserver.authz.AuthorizerImpl.authorize(Authorizer.scala:393)
at com.amazonaws.emr.recordserver.handler.AuthorizationHandler.$anonfun$channelRead0$1(AuthorizationHandler.scala:33)
at com.amazonaws.emr.recordserver.metrics.Metrics$.withMetering(Metrics.scala:49)
at com.amazonaws.emr.recordserver.handler.AuthorizationHandler.channelRead0(AuthorizationHandler.scala:33)
```

### Kyuubi Engine Log Output

_No response_

### Kyuubi Server Configurations

_No response_

### Kyuubi Engine Configurations

_No response_

### Additional context

_No response_

### Are you willing to submit PR?

- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [X] No. I cannot submit a PR at this time.

Contributor guide

Open the contributing guide

Research direction

Start with the KyuubiServer entry point and the supplied server command, then trace the EMR recordserver AuthorizationException in AuthorizerImpl and AuthorizationHandler. Reproduce the beeline connection with the shown principal and verify that the hadoop user no longer receives a permission-denied error for the default database.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, hadoop, spark
Domain
authorization, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.