Undocumented distributed inference behaviour for `run_summarization.py`

### System Info

```shell
Fails with error

Traceback (most recent call last):
File "/scratches/neuron/anaconda3/envs/T5DST-SGD/bin/transformers-cli", line 5, in
from transformers.commands.transformers_cli import main
File "/scratches/neuron/anaconda3/envs/T5DST-SGD/lib/python3.8/site-packages/transformers/commands/transformers_cli.py", line 26, in
from .user import UserCommands
File "/scratches/neuron/anaconda3/envs/T5DST-SGD/lib/python3.8/site-packages/transformers/commands/user.py", line 20, in
from huggingface_hub.hf_api import HfFolder, create_repo, list_repos_objs, login, logout, whoami
ImportError: cannot import name 'list_repos_objs' from 'huggingface_hub.hf_api' (/scratches/neuron/anaconda3/envs/T5DST-SGD/lib/python3.8/site-packages/huggingface_hub/hf_api.py)

However I am running `4.16.2` with python `3.8`.
```

### Who can help?

@sgugger @stevhliu @patil-suraj

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

I am working with a copy of the `run_summarization.py` (pytorch) example that the authors of this [paper](https://arxiv.org/pdf/2109.07506.pdf) modified to work for dialogue state tracking (implemented [here ](https://github.com/chiahsuan156/DST-as-Prompting) for reference)

The `run_summarization.py` script can be launched with `torch.distributed.launch` and the `--do_predict` option. This shards the examples in the test set to various GPUs and therefore generation and task-oriented metrics is accelerated. The predictions are written to the `generated_predictions.txt` file in the output directory.

To be able to compute dialogue-relevant task oriented metrics, one ought to run a postprocessing script that uses the `generated_predictions.txt`. Because the trainer erases all the columns that are not keys to the model `forward` method from the dataset, the metadata that informs us of what training examples the predictions are related to is lost. Therefore, we rely on the ordering of the `generated_predictions.txt` to match the order of the examples in the dataset.

My question is:
- Does `predictions` (`L675`) obey the order of the `dataset`? So if my dataset has 1m examples, will the 1m entries in the `predictions` list match the order of the dataset iterator? In my experience this depends on implementation* and the behaviour is not documented.

*For example, in frameworks such as `ray` you have to explicitly enforce the order in which the results are returned and the predictions may be returned out of order - if a process finishes, it returns its results so it can be given more work by an external load balancer.

### Expected behavior

```shell
Improved documentation about expected behaviour here. Happy to discuss where this should be added and contribute a small PR to clarify this important issue.
```

Start with the copied run_summarization.py flow around L675, then trace how torch.distributed.launch and Trainer produce generated_predictions.txt. Verify whether prediction ordering matches the dataset under distributed --do_predict, including the metadata loss described in the issue. Done means documenting the observed ordering behavior and its relevant limitations.

Written by the indexing model from the issue text.