Chart scheduler persistence should not depend on celery configuration
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Under which category would you file this issue?
Helm chart
### Apache Airflow version
main
### What happened and how to reproduce it?
I noticed what the logic to determine if a scheduler is a deployment or a statefulset now looks at the `.Values.workers.celery.persistence.enabled `. This used to be the more general `.Values.workers.persistence.enabled`, but now this coupling does not make sense.
This was a miss when celery was split out of the general workers section.
### What you think should happen instead?
I haven't spent much time thinking about it, but maybe we just need a new `scheduler.persistence` to control it? Or `workers.local.persistence`?
Whoever picks this up should explore the possible options.
### Operating System
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
main (development)
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Trace the Helm chart logic that decides whether the scheduler is a Deployment or StatefulSet, starting with its use of `.Values.workers.celery.persistence.enabled`. Compare that coupling with the former `.Values.workers.persistence.enabled`, then evaluate where an independent scheduler persistence setting should live; done means scheduler persistence no longer depends on Celery configuration and the selected behavior is consistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100