apache / apache/airflow

Add support for other Keda Scalers

Open
#34,863 1 comment 1 reaction 0 assignees View on GitHub
area:helm-chart kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Right now the Official Charts have support for the Keda scaler but it is limited using `postgresql` as the source of the trigger. This works great for most use cases, but sometimes it might be required to use some other scaler (for example a Prometheus metric).
Technically we could add a `ScaledObject` outside of this chart to handle this but the `replicas` field in the Workers depends on whether or not you are using the charts' one, so there are conflicts in the replica count when trying to deploy.

To fix it, we could either move the current configuration to the values so it can be overwritten if desired or decoupling the `replicas` field in the worker from whether `keda` is being used or not.

### Use case/motivation

_No response_

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by locating the Official Chart's current KEDA ScaledObject configuration and the Worker replicas setting. Compare the PostgreSQL trigger path with the requested Prometheus-style scaler support, then decide whether values should expose the configuration or replicas should be decoupled; done means alternate scaler configurations deploy without replica-count conflicts.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes, postgresql, prometheus
Domain
devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.