[testing] data size / dynamic downloads - test speed and repo bloat

Let's discuss which data is used in the test suite. And after the discussion turn into guidelines for test writers.

Here is a very rough start:

* We want to have the basic test suite run really fast. I have started curating tiny datasets and tokenizers sufficient for testing here:
https://github.com/bigscience-workshop/Megatron-DeepSpeed/tree/main/tests/data/gpt2
So the normal tests ideally should complete fast.

Currently the main overhead in testing is the really slow startup of Meg-DS

* And we also don't want the repo to become slow to download because of large data files. My first attempt is to generate a synthetic input on the fly (incomplete). But perhaps have a small compressed file in the repo should not be a problem and having it uncompressed when the test suite starts.

Hopefully soon we will have a CI, so downloading large data files can slow things down.

The test suite ensures correctness, not speed, so having 1k records is more than enough, even for parallel processing.

That's said we can have extended slow tests that don't run by default and which can download large data and then these can be used as well.

Comments, suggestions and ideas are super welcome.