PodAutoscaler resource not found. Clean scaler object in memory since object default/llama2-70b-pa must have been deleted

### 🐛 Describe the bug

```
controllerKind"="ModelAdapter" "worker count"=1
I0816 03:00:35.848383 1 controller.go:217] "msg"="Starting workers" "controller"="kv-cache-controller" "controllerGroup"="orchestration.aibrix.ai" "controllerKind"="KVCache" "worker count"=1
....
I0816 03:00:35.850500 1 podautoscaler_controller.go:202] PodAutoscaler resource not found. Clean scaler object in memory since object default/llama2-70b-pa must have been deleted
```

```
k get podautoscaler --all-namespaces
No resources found
```

### Steps to Reproduce

just deploy aibrix from scratch

### Expected behavior

it should not have throw such warning messages in clean cluster

### Environment

nightly