Airflow Helm Chart: Scheduler Should Support Full Probe Configuration (livenessProbe/readinessProbe/readinessProbe)
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Official Helm Chart version
1.18.0 (latest released)
### Apache Airflow version
3.0.2 (affects any)
### Kubernetes Version
1.34 (affects any)
### Helm Chart configuration
```yaml
scheduler:
livenessProbe:
initialDelaySeconds: 20
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# Attempt to set a command, or configure as TCP/HTTP (see below attempts)
# command: ["airflow", "jobs", "check", "--job-type", "SchedulerJob", "--local"]
# Attempted to add:
# readinessProbe:
# tcpSocket:
# port: 8793
```
### Docker Image customizations
None. Using the official Apache Airflow image.
### What happened
I attempted to enable and customize both liveness and readiness probes for the scheduler pod. However, the `scheduler` section of the Helm values only permits a very limited set of probe fields. The only keys allowed are the timing parameters and `command` for the liveness probe. Attempts to add typical Kubernetes probe types (like `tcpSocket`, `httpGet`, or even `readinessProbe:` at all) result in schema validation errors during `helm template` or `install`, e.g.:
```
Error: values don't meet the specifications of the schema(s) in the following chart(s):
airflow:
- scheduler: Additional property readinessProbe is not allowed
- scheduler.livenessProbe: Additional property tcpSocket is not allowed
```
### What you think should happen instead
Both `livenessProbe` and `readinessProbe` should support the full set of [Kubernetes probe options](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) (such as `exec`, `tcpSocket`, `httpGet`, etc.) for the scheduler, as is already the case with other components in the chart (like workers and webserver). Ideally, there should also be a `readinessProbe` supported for `scheduler` the same way as with the other pods.
### How to reproduce
1. Try to set a readiness probe for the scheduler in `values.yaml`:
```yaml
scheduler:
readinessProbe:
tcpSocket:
port: 8793
```
2. Or try to set a non-default liveness probe for the scheduler:
```yaml
scheduler:
livenessProbe:
tcpSocket:
port: 8793
initialDelaySeconds: 10
```
3. Run:
```bash
helm template my-airflow apache-airflow/airflow -f values.yaml
```
4. You get schema validation errors:
```
scheduler: Additional property readinessProbe is not allowed
scheduler.livenessProbe: Additional property tcpSocket is not allowed
```
### Anything else
Every time I try to set scheduler probe fields beyond the very limited default, I hit this error. The issue occurs consistently on new chart pulls as well.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the scheduler probe settings in values.yaml and reproduce the schema errors with helm template using the examples in the issue. Compare the scheduler configuration with the worker and webserver probe configuration described in the report. Done means scheduler livenessProbe and readinessProbe accept the requested Kubernetes probe options without schema validation errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100