Feature request: support for multi-node, multi-GPU distributed inference

### Motivation

Currently, device_map="auto" only support a single-node, multi-GPU setup (https://github.com/huggingface/transformers/issues/24747). If you have access to 8xA100 80GB/40GB, things are mostly fine, but not everyone has access to such a setup. Therefore, I believe there is benefit to support multi-node, multi-gpu inference. For example, if someone has access to multi-node 4xV100 setup but not 8xA100, then this feature would allow them to experiment large models like Llama-2-80B on their multi-node 4xV100 setup. As far as I know, there doesn't appear to be a straightforward method for running inference with models of the scale of Llama-2-80B on multi-node V100 setups.