ALBEF: Align Before Fuse

### Model description

Align Before Fuse (ALBEF) is a vision-language (VL) model that showed competitive results in numerous VL tasks such as image-text retrieval, visual question answering, visual entailment, and visual grounding.

The authors propose to use text encoder (BERT's first half layers) and image encoder (ViT) to create an aligned representation for respective modality before fusing them together with a multi-modal encoder (BERT's second half layers). The model is trained on multi-modal representation tasks and momentum distillation to achieve state-of-the-art results in VL tasks.

As multi-modal models are gaining more attention in academia/industry, I think ALBEF could be a nice addition to the transformers library.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

- There are an official implementation and pre-trained/fine-tuned weights by the authors at this [repo](https://github.com/salesforce/ALBEF)
- Link to the [paper](https://arxiv.org/abs/2107.07651)

Start by reviewing the official Salesforce/ALBEF implementation and the linked paper to understand the proposed model and available weights. Then identify the Transformers integration entry points and relevant multimodal model tests. Done means ALBEF is added to the library with its implementation and pretrained weights working for the stated vision-language tasks.

Written by the indexing model from the issue text.