apache / apache/dolphinscheduler

[Improvement][Helm Chart] Support HDFS HA

Open
#16,390 0 comments 0 reactions 0 assignees View on GitHub
help wanted improvement kubernetes
Dominant language
Java
Stars
14.5k
Forks
5.1k
Avg merge
1d 21h
Merged PRs (30d)
28

Description

### Search before asking

- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.

### Description

The helm chart support HDFS HA.

### Use case

Now, the config option has "resource.hdfs.fs.defaultFS".
And the comment say "if resource.storage.type=HDFS and namenode HA is enabled, you need to copy core-site.xml and hdfs-site.xml to conf dir", but the template not use the "conf" dir or "config" dir.
The pod only mount common.properties.

I want some ways to support HDFS enabled HA. Maybe some option for the xml files in common.properties, or some mount.

And, i m using HDFS deployed by stackable hdfs operator, the core-site.xml and hdfs-site.xml are ConfigMaps. Maybe some way can use this.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the Helm chart template and pod volume mounts around common.properties, especially the conf or config directories. Review how core-site.xml and hdfs-site.xml are expected to be supplied, including ConfigMaps from the Stackable HDFS operator. Done means the chart provides a documented way to configure HDFS HA and makes both XML files available to the pod.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, helm, kubernetes
Domain
data-engineering, devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.