[Benchmark] HF Trainer on RTX-3090

# 🖥 Benchmarking `transformers` w/ HF Trainer on RTX-3090

We are going to use a special benchmarking tool that will do all the work for us. https://github.com/huggingface/transformers/pull/14934

This is the index post and specific benchmarks are in their own posts below:

1. [fp16 vs bf16 vs tf32 vs fp32](https://github.com/huggingface/transformers/issues/14608#issuecomment-1004390803)
2. [gradient accumulation steps](https://github.com/huggingface/transformers/issues/14608#issuecomment-1004392537)
3. [gradient checkpointing](https://github.com/huggingface/transformers/issues/14608#issuecomment-1004422281)
4. [batch size](https://github.com/huggingface/transformers/issues/14608#issuecomment-1004392537)
5. [optimizers](https://github.com/huggingface/transformers/issues/14608#issuecomment-1005219385)
6. [combining winning strategies](https://github.com/huggingface/transformers/issues/14608#issuecomment-1005229426) **~2x speed improvement!**
7. [RTX-3090 vs A100](https://github.com/huggingface/transformers/issues/15026#issuecomment-1005235845)

See also the [same benchmarks for A100](https://github.com/huggingface/transformers/issues/15026)

TODO:
- other suggestions?

Note that each benchmark was run only once, so multiple runs and averaging is probably going to give slightly different results. The purpose here though is to see relative differences roughly and not try to give an exact number.