4paradigm / 4paradigm/OpenMLDB
sbin deploy can't set HADOOP_CONF_DIR for taskmanager
- 主要语言
- C++
- 星标
- 1.7k
- 派生
- 331
- 平均合并
- 12 天 12 小时
- 30 天内合并 PR
- 1
描述
and spark extra conf, e.g. hdfs-site.xml, needs to add in spark/conf before deploy. -- HADOOP_CONF_DIR设置为openmldb/conf应该可以?
但taskmanager读取需要start.sh时有环境,sbin不会传去host里
just deploy conf for fix config options
More
stat-all.sh没有recover工具时最好failed,除非force执行,不然用户很容易重启后表有问题,但当时无法察觉
start-all.sh里的recovering没有进度,前台也没法看,没有多开shell时很麻烦
recovering本来就是在tablet、nameserver都启动后的操作,在线集群可以独立recovering,不做不必要的stop
yarn job如果日志出现connect 0.0.0.0:8032,说明yarn-site.xml没有正确load。使用了默认yarn地址。
贡献指南
调研方向
Look at sbin deploy scripts to see how environment variables like HADOOP_CONF_DIR are passed to taskmanager. Examine start.sh and related host deployment logic. The fix likely involves modifying deployment scripts to propagate configuration. Check spark/conf handling for hdfs-site.xml. Also review stat-all.sh and start-all.sh for recovery behavior improvements mentioned in the issue.
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- hadoop, shell, spark
- 领域
- backend, cli, devops
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 45/100