Loading repository after rename does not work (with old name)

### System Info

```shell
- `transformers` version: 4.19.2
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- Huggingface_hub version: 0.7.0
- PyTorch version (GPU?): 1.11.0+cu113 (False)
- Tensorflow version (GPU?): 2.8.2 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?:
- Using distributed or parallel set-up in script?:
```

### Who can help?

cc @LysandreJik

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Context
```
FROM LOCATION (old location): https://huggingface.co/KB/bert-base-swedish-cased
TO LOCATION (new location): https://huggingface.co/KBLab/bert-base-swedish-cased
```

The following breaks
```py
from transformers import AutoModel
model = AutoModel.from_pretrained("KB/bert-base-swedish-cased")
```

Error
```
OSError: Can't load config for 'KB/bert-base-swedish-cased'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'KB/bert-base-swedish-cased' is the correct path to a directory containing a config.json file
```

What works here?
:white_check_mark: going to original path redirects to new path
:white_check_mark: doing git clone
:white_check_mark: using huggingface_hub model_info("KB/bert-base-swedish-cased")
:x: using transformers AutoModel.from_pretrained("KB/bert-base-swedish-cased")

### Expected behavior

```shell
Loading the model with the old name would still work so users loading a transferred model don't have an impacted experience
```

Start by reproducing the failure at AutoModel.from_pretrained("KB/bert-base-swedish-cased") and trace the repository-loading path used by Transformers with huggingface_hub. Compare it with model_info and git clone behavior; done means the old repository identifier loads the transferred model and a regression test covers the rename case.

Written by the indexing model from the issue text.