[Dist KV] vllm pods which do not have kvcache pods running in the same node crashes.

### 🚀 Feature Description and Motivation

vllm pods which do not have kvcache pods running in the same node crashes.

All vllm pods should run with kvcache pod in the same node.

Temporary solution would be making kvcache pods spread in all nodes using affinity and antiaffinity. but it is not too unreliable. More elegant and reliable solution is needed.

### Use Case

distributed kv cache set up

### Proposed Solution

_No response_