[Helm Chart] Support multiple DagProcessors
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
Airflow 2.4.0 has introduced support for [Support multiple DagProcessors parsing files from different locations](https://github.com/apache/airflow/pull/25935).
However, Helm chart only supports [Provision Standalone Dag Processor](https://github.com/apache/airflow/pull/23711) (or at least will support in Airflow Helm Chart 1.7.0).
### Use case/motivation
For Airflow Helm Chart users take advantage of these new [AIP-43](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-43+DAG+Processor+separation) features, it is necessary to add support for provision multiple DagProcessors in Helm chart.
### Related issues
[Helmchart: Provision Standalone Dag Processor](https://github.com/apache/airflow/pull/23711)
[Support multiple DagProcessors parsing files from different locations](https://github.com/apache/airflow/pull/25935)
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the Helm chart's existing standalone Dag Processor configuration and read the linked Airflow 2.4.0 multiple-DagProcessor change. Done means the chart can provision multiple DagProcessors so users can parse files from different locations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100