4paradigm / 4paradigm/OpenMLDB
Fail to stop yarn job when kerberos is enabled
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 331
- Avg merge
- 12d 12h
- Merged PRs (30d)
- 1
Description
**Bug Description**
We create the yarn client without initializing with keytab which may be work.
```
def createYarnClient(): YarnClient = {
val config = new Configuration()
config.addResource(new Path(TaskManagerConfig.getHadoopConfDir, "core-site.xml"))
config.addResource(new Path(TaskManagerConfig.getHadoopConfDir, "hdfs-site.xml"))
config.addResource(new Path(TaskManagerConfig.getHadoopConfDir, "yarn-site.xml"))
config.addResource(new Path(TaskManagerConfig.getHadoopConfDir, "mapred-site.xml"))
// Create yarn client
val yarnClient = YarnClient.createYarnClient()
yarnClient.init(config)
yarnClient.start()
yarnClient
}
```
Contributor guide
Research direction
Look at the Yarn client creation code in the issue, likely in a Scala file related to task management. Understand how Kerberos authentication works with Yarn. Check the project's configuration loading for keytab files. The fix involves initializing the Yarn client with proper Kerberos credentials. Test by running a Yarn job with Kerberos enabled to verify the stop functionality works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, scala
- Domain
- backend, databases, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100