apache / apache/dolphinscheduler
[Improvement][Helm Chart] Support HDFS HA
- Dominant language
- Java
- Stars
- 14.5k
- Forks
- 5.1k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 28
Description
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
The helm chart support HDFS HA.
### Use case
Now, the config option has "resource.hdfs.fs.defaultFS".
And the comment say "if resource.storage.type=HDFS and namenode HA is enabled, you need to copy core-site.xml and hdfs-site.xml to conf dir", but the template not use the "conf" dir or "config" dir.
The pod only mount common.properties.
I want some ways to support HDFS enabled HA. Maybe some option for the xml files in common.properties, or some mount.
And, i m using HDFS deployed by stackable hdfs operator, the core-site.xml and hdfs-site.xml are ConfigMaps. Maybe some way can use this.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start by inspecting the Helm chart template and pod volume mounts around common.properties, especially the conf or config directories. Review how core-site.xml and hdfs-site.xml are expected to be supplied, including ConfigMaps from the Stackable HDFS operator. Done means the chart provides a documented way to configure HDFS HA and makes both XML files available to the pod.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, helm, kubernetes
- Domain
- data-engineering, devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100