Model orchestration with heterogeneous hardwares

We meet a few cases that single deployment needs to be deployed across different chips due to quota or resource shortage. However, in Kubernetes, most of the time we use `Deployment` to manage a group of pods using one type of GPU, If we remove GPU type constraints, then it's hard to control the ratio. Technically, we can workaround the problem using multiple deployment, but the rolling upgrade control additional control, same as HPA. The RoleSet CRD is not able to manage the such cases as well.

1. We may need other orchestrators for instances using heterogeneous hardwares, HPA, Rolling upgrade need to be revised as well.
2. We need more advanced Traffic Routing solutions to handle such differences
3. It also brings lots of challenges on monitoring at the service level etc