apache / apache/airflow

scheduler, triggerer, worker airflow components have inconsistent log volume

Open
#56,015 1 comment 0 reactions 0 assignees View on GitHub
area:helm-chart kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Official Helm Chart version

1.18.0 (latest released)

### Apache Airflow version

initContainers in scheduler, triggerer use inconsistent log volume configs

### Kubernetes Version

1.32

### Helm Chart configuration

_No response_

### Docker Image customizations

_No response_

### What happened

In the scheduler, worker, and triggerer, the `wait-for-airflow-migrations` initContainer uses inconsistent `/usr/local/airflow/log` configurations (none) while the main containers uses a tailored `/usr/local/airflow/log` configuration. This creates an inconsistency between containers that run airflow. In my specific use case, this is preventing me from adding a container `securityContext.readOnlyRootFilesystem: true` because I am unable to create an `emptyDir` for `/usr/local/airflow/logs` for the airflow components to write to. If I set that config, I can't deploy because that volume already exists in the `scheduler` container, but if I do not set it, the `wait-for-airflow-migrations` fails because the logs dir is read-only.

### What you think should happen instead

All components in a pod that run airflow should have consistent volume handling.

### How to reproduce

Install with these extra values:

```
scheduler:
extraVolumeMounts:
- mountPath: /usr/local/airflow/logs
name: airflow-logs
- mountPath: /tmp
name: tmp
extraVolumes:
- emptyDir: {}
name: airflow-logs
- emptyDir: {}
name: tmp
securityContexts:
containers:
readOnlyRootFilesystem: true
```

Also try it with the `airflow-logs` volume and volumeMount removed.

### Anything else

I am not reporting any problems with readOnlyRootFilesystem, just the volume problem, which was found when configuring readOnlyRootFilesystem.

* initContainer volume: https://github.com/apache/airflow/blob/main/chart/templates/scheduler/scheduler-deployment.yaml#L150-L165
* the missing configs: https://github.com/apache/airflow/blob/main/chart/templates/scheduler/scheduler-deployment.yaml#L289-L303

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start with chart/templates/scheduler/scheduler-deployment.yaml, especially the initContainer volume block at lines 150-165 and the missing configuration around lines 289-303. Compare volume handling for the scheduler, worker, and triggerer initContainers and main containers using the reproduction values. Done means all components that run Airflow handle the log volume consistently and the readOnlyRootFilesystem deployment succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.