vllm-project / vllm-project/aibrix

[Code] Remove excessive logs when "model.aibrix.ai/port" is not available by default and improve efficiency

Open
#1,179 1 comment 0 reactions 0 assignees View on GitHub
area/gateway kind/enhancement
Dominant language
Go
Stars
5.1k
Forks
697
Avg merge
1d 19h
Merged PRs (30d)
104

Description

Currently, the gateway outputs excessive warning logs when the "model.aibrix.ai/port" label is not available. The log is output by the following code in pkg/util/pod.go.

```
func GetModelPortForPod(requestID string, pod *v1.Pod) int64 {
value, ok := pod.Labels[modelPortIdentifier]
if !ok {
klog.Warningf("requestID: %v, pod: %v is missing port identifier label: %v, hence default to port: %v",
requestID, pod.Name, modelPortIdentifier, defaultPodMetricPort)
return defaultPodMetricPort
}

modelPort, err := strconv.ParseInt(value, 10, 32)
if err != nil {
klog.Warningf("requestID: %v, pod: %v has incorrect value: %v for port identifier label: %v, hence default to port: %v",
requestID, pod.Name, value, modelPortIdentifier, defaultPodMetricPort)
modelPort = defaultPodMetricPort
}
return modelPort
}
```

This code leads to two problems:
1. Excessive warning logs if the label is missing.
2. Inefficiency if the label is available because the string will be parsed on a per-request basis.

Contributor guide

Open the contributing guide

Research direction

Start in pkg/util/pod.go at GetModelPortForPod and inspect its callers to understand why it runs per request. The change is done when pods without the label no longer produce excessive warning logs and an available label is not parsed on every request; the issue does not name a test to run.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
backend, performance
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.