4paradigm / 4paradigm/OpenMLDB

Fail to stop yarn job when kerberos is enabled

Open
#3,671 0 comments 0 reactions 1 assignee Claimed by @tobegit3hub View on GitHub
bug high-priority
Dominant language
C++
Stars
1.7k
Forks
331
Avg merge
12d 12h
Merged PRs (30d)
1

Description

**Bug Description**

We create the yarn client without initializing with keytab which may be work.

```
def createYarnClient(): YarnClient = {
val config = new Configuration()
config.addResource(new Path(TaskManagerConfig.getHadoopConfDir, "core-site.xml"))
config.addResource(new Path(TaskManagerConfig.getHadoopConfDir, "hdfs-site.xml"))
config.addResource(new Path(TaskManagerConfig.getHadoopConfDir, "yarn-site.xml"))
config.addResource(new Path(TaskManagerConfig.getHadoopConfDir, "mapred-site.xml"))

// Create yarn client
val yarnClient = YarnClient.createYarnClient()
yarnClient.init(config)
yarnClient.start()

yarnClient
}
```

Contributor guide

Open the contributing guide

Research direction

Look at the Yarn client creation code in the issue, likely in a Scala file related to task management. Understand how Kerberos authentication works with Yarn. Check the project's configuration loading for keytab files. The fix involves initializing the Yarn client with proper Kerberos credentials. Test by running a Yarn job with Kerberos enabled to verify the stop functionality works.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, scala
Domain
backend, databases, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.