Use tiny models for the tests

### 🚀 The feature, motivation, and pitch

Using tiny models for the tests may speed up the tests by a factor of 2 or 3, while still effectively verifying the functionalities.

Some examples of tiny models worth considering:
- google/t5-efficient-tiny (Seq2Seq)
- sshleifer/tiny-gpt2 (has perhaps some numerical instabilities, but it should not matter for most tests)
- EleutherAI/pythia-70m (not so tiny, but still smaller than EleutherAI/pythia-160m that is currently used)
- roneneldan/TinyStories-1M (particularly small)