✨ DataParallel and DistributedDataParallel for speed up training.

DataParallel: multiple thread for single machine multiple GPUs
- unbalance GPU memory and GPU usage. ([discuss.pytorch.org: Use `FullModel` which writes loss function into the model to solve the memory usage imbalance problem. ](https://discuss.pytorch.org/t/dataparallel-imbalanced-memory-usage/22551/6))
- slow
- Collecting gradients by a serial method

DistributedDataParallel: multiple processing for single or multiple machines and multiple GPUs.
- balance GPU memory and GPU usage. (don't need to use `FullModel`)
- faster than DataParallel
- [Ring-Allreduce by pytorch](https://pytorch.org/tutorials/intermediate/dist_tuto.html#our-own-ring-allreduce)

It is very easy to add **DataParallel** into the code, but DataParallel brings less speed up.

It's a little tricky to use because **DistributedDataParallel** needs to be started from the command line, but it gives a significant speedup with 4 GPUs in single machine in high GPU memory.