[RFC] Enable node level autoscaling with pod autoscaling coordination

### Summary

We propose enhancing the autoscaling system to enable node-pod autoscaling coordination, particularly critical in GPU-constrained environments like large language model (LLM) scenarios. This system would handle cases where scaling decisions are not feasible due to infrastructure capacity limitations, providing feedback and enabling the system to quickly choose alternative or sub-optimal solutions. Currently, this type of feedback loop and coordination is not supported by traditional Kubernetes autoscalers.

### Motivation

In large-scale LLM scenarios, GPU resources are highly limited and often constrained. Autoscaling decisions made at the pod level may fail to be executed at the IaaS layer due to a lack of available resources. This leads to inefficiencies in scaling behavior, causing service disruptions or poor resource utilization. The system needs a way to:

1. Provide feedback when capacity is unavailable at the node level.
2. Allow the autoscaling decider to react quickly and select alternative or sub-optimal scaling options, such as scaling across different nodes or adjusting the pod resource requests.

This coordination between pod-level autoscaling and node availability is essential for better managing limited resources like GPUs and ensuring smoother operation in resource-constrained environments.

### Proposed Change

1. Implement a node-pod coordination system that allows feedback from the IaaS layer to the autoscaler when capacity constraints prevent the execution of scaling decisions.
2. Provide mechanisms for the autoscaler to adjust scaling strategies dynamically based on this feedback. For example, the autoscaler could switch to scaling with alternative resource types or adjust pod resource requests to fit within available capacity.
3. Improve coordination to allow sub-optimal or fallback scaling decisions to be made quickly, ensuring that services remain operational even when optimal scaling isn't possible.
4. Incorporate this logic into the custom autoscaler for handling LLM scenarios and GPU resource limitations efficiently.

We should leverage karpenter's API and our own pod autoscaler for integration.

### Alternatives Considered

_No response_

No files, tests, or entry points are named. Start by locating the custom pod autoscaler and its Karpenter integration, then define how IaaS capacity feedback reaches scaling decisions and how GPU-constrained fallback choices are evaluated; done means coordinated node-pod scaling with fast alternative decisions.

Written by the indexing model from the issue text.