Kafka functions: KEDA scaling with consumer-lag trigger
- Dominant language
- Go
- Stars
- 365
- Forks
- 223
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 25
Description
`func deploy` already supports `--deployer keda` for KEDA-based scaling. For Kafka functions, add a Kafka consumer-lag trigger so KEDA can scale based on consumer group lag — enabling scale-to-zero when there are no messages and scaling up under load. Currently Kafka functions require `min-scale: 1` to stay running.
Traffic-based autoscaling (e.g. Knative's KPA or KEDA HTTP scaler) does not work for Kafka functions because there is no inbound HTTP traffic — the function pulls messages from Kafka internally. Scaling must be driven by Kafka consumer group lag instead.
Contributor guide
Research direction
Start at the func deploy entry point for --deployer keda and trace how Kafka functions are represented in the deployment configuration. Compare the existing KEDA path with the Kafka consumer-group settings; done means KEDA can use consumer-group lag to scale Kafka functions to zero and up under load.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka, kubernetes
- Domain
- backend, cli, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100