GPU Isolation and flexible deployment strategies [FEA]

Is your feature request related to a problem? Please describe.
If we consider a few scenarios where we need

• to deploy multiple models for a single application.
• deploy multiple models on the same machine with different GPU architectures.
• lockin resources for deployment so I can do training with the remaining resources.

In all these examples, we want to assign a GPU to a model and do not want the inference service to take up the entire system. If we can isolate the GPU and pin it to a particular deployment, it will be really useful. In addition, this will also future proof our deployments. Imagine a scenario where we get new GPUs with new architectures. Maybe the deployment and the model and pytorch versions do not work with the new architecture. In such a case, we can add more GPUs without disturbing the deployments.

Describe alternatives you've considered
@slbryson has tried GPU isolation using clara CLI tools.

Additional context