scheduler, triggerer, worker airflow components have inconsistent log volume
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Official Helm Chart version
1.18.0 (latest released)
### Apache Airflow version
initContainers in scheduler, triggerer use inconsistent log volume configs
### Kubernetes Version
1.32
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### What happened
In the scheduler, worker, and triggerer, the `wait-for-airflow-migrations` initContainer uses inconsistent `/usr/local/airflow/log` configurations (none) while the main containers uses a tailored `/usr/local/airflow/log` configuration. This creates an inconsistency between containers that run airflow. In my specific use case, this is preventing me from adding a container `securityContext.readOnlyRootFilesystem: true` because I am unable to create an `emptyDir` for `/usr/local/airflow/logs` for the airflow components to write to. If I set that config, I can't deploy because that volume already exists in the `scheduler` container, but if I do not set it, the `wait-for-airflow-migrations` fails because the logs dir is read-only.
### What you think should happen instead
All components in a pod that run airflow should have consistent volume handling.
### How to reproduce
Install with these extra values:
```
scheduler:
extraVolumeMounts:
- mountPath: /usr/local/airflow/logs
name: airflow-logs
- mountPath: /tmp
name: tmp
extraVolumes:
- emptyDir: {}
name: airflow-logs
- emptyDir: {}
name: tmp
securityContexts:
containers:
readOnlyRootFilesystem: true
```
Also try it with the `airflow-logs` volume and volumeMount removed.
### Anything else
I am not reporting any problems with readOnlyRootFilesystem, just the volume problem, which was found when configuring readOnlyRootFilesystem.
* initContainer volume: https://github.com/apache/airflow/blob/main/chart/templates/scheduler/scheduler-deployment.yaml#L150-L165
* the missing configs: https://github.com/apache/airflow/blob/main/chart/templates/scheduler/scheduler-deployment.yaml#L289-L303
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with chart/templates/scheduler/scheduler-deployment.yaml, especially the initContainer volume block at lines 150-165 and the missing configuration around lines 289-303. Compare volume handling for the scheduler, worker, and triggerer initContainers and main containers using the reproduction values. Done means all components that run Airflow handle the log volume consistently and the readOnlyRootFilesystem deployment succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100