knative / knative/serving

Feature-Request: add memory utilization to activator-hpa

Open
#13,843 11 comments 2 reactions 0 assignees View on GitHub
kind/enhancement triage/accepted
Dominant language
Go
Stars
6.1k
Forks
1.2k
Avg merge
2d 7h
Merged PRs (30d)
2

Description

## Context
Currently we set activator-hpa only based on a `cpu` [target](https://github.com/knative/serving/blob/main/config/core/deployments/activator-hpa.yaml#L34). As the activator is potentially buffering a lot of requests, it can happen that the configured memory is not sufficient, causing the activator to be OOM-killed:

```
kubectl describe pod -n knative-serving activator-774d4ff4b8-4l5vp | grep Reason
Reason: OOMKilled
```

Obviously, the configured resources should be adapted depending on the expected workload (small cluster, big cluster, amount of `Kservices`, traffic payload sizes etc.), but we also could add a `memory` target to the activator-hpa to automatically scale more instances of activator if `memory` pressure increases.

## Describe the feature
- Add a `memory` target to [activator-hpa](https://github.com/knative/serving/blob/main/config/core/deployments/activator-hpa.yaml)
- Add some docs around setting resource limits based on workload sizes
- Add some docs around potential issues (like [this](https://github.com/knative/serving/issues/13583#issuecomment-1469160313)) and recommendations (like setting TBC to 0).

Related: https://github.com/knative/serving/issues/13583

/area autoscale

Contributor guide

Open the contributing guide

Research direction

Start with config/core/deployments/activator-hpa.yaml and review the existing CPU target. Read the linked issue 13583 and its referenced comment for the potential issues and recommendations, then identify where workload-based resource guidance belongs. Done means the activator HPA includes a memory target and documentation covers resource limits, risks, and recommendations such as setting TBC to 0.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.