fastai parity

This issue tracks the progress on fastai parity.

_Last updated 2021/08/11_

### Datasets

- [x] easily download datasets from fastai dataset collection ([`datasetpath`](https://fluxml.ai/FastAI.jl/dev/REFERENCE/FastAI.Datasets.Datasets.datasetpath.html))
- [x] flexible APIs for loading datasets in varied formats ([Data container tutorial](https://fluxml.ai/FastAI.jl/dev/docs/data_containers.md.html))

### Data pipelines

- [x] create data pipelines from data block information (`BlockMethod`)
- [x] visualizations based on blocks
- [x] fast, paralllelized data loading ([DataLoaders.jl](https://github.com/lorenzoh/DataLoaders.jl))
- [x] fast, composable affine augmentations for images, masks and keypoints ([DataAugmentation.jl](https://github.com/lorenzoh/DataAugmentation.jl))
- [ ] on GPU: GPU support is still WIP, see https://github.com/lorenzoh/DataAugmentation.jl/issues/48
- advanced data augmentation
- [ ] MixUp
- [ ] CutMix

### Models

- architectures
- [x] XResNet ([FastAI.Models.xresnet*](https://fluxml.ai/FastAI.jl/dev/REFERENCE/FastAI.Models.Models.xresnet18.html))
- [x] UNet ([FastAI.Models.UNetDynamic](https://fluxml.ai/FastAI.jl/dev/REFERENCE/FastAI.Models.Models.UNetDynamic.html))
- [x] pretrained weights ([Metalhead.jl#70](https://github.com/FluxML/Metalhead.jl/pull/70))

### Training

- training schedules
- [x] hyperparameter scheduling ([Hyperparameter scheduling in FluxTraining.jl](https://fluxml.ai/FluxTraining.jl/dev/docs/tutorials/hyperparameters.md.html))
- [x] one-cycle schedule [One-cycle in FastAI.jl](https://fluxml.ai/FastAI.jl/dev/notebooks/fitonecycle.ipynb.html))
- [x] finetuning schedule [Finetuning in FastAI.jl](https://fluxml.ai/FastAI.jl/dev/notebooks/finetune.ipynb.html))
- [ ] flat cosine schedule
- [ ] mixed precision training

AFAIK this is currently in the works by the Flux.jl team

- [ ] distributed data parallel training

Lots of progress at [DaggerFlux.jl](https://github.com/DhairyaLGandhi/DaggerFlux.jl). Needs to be integrated with FastAI.jl

- callbacks
- [x] early stopping ([`EarlyStopping`](https://fluxml.ai/FluxTraining.jl/dev/REFERENCE/FluxTraining.EarlyStopping.html))
- [x] checkpointing ([`Checkpointer`](https://fluxml.ai/FluxTraining.jl/dev/REFERENCE/FluxTraining.Checkpointer.html))

Works but could use some improvements such as conditional checkpointing

- [x] stopping on NaN loss ([`StopOnNaNLoss`](https://fluxml.ai/FluxTraining.jl/dev/REFERENCE/FluxTraining.StopOnNaNLoss.html))
- [x] metrics and history tracking ([`Metrics`](https://fluxml.ai/FluxTraining.jl/dev/REFERENCE/FluxTraining.Metrics.html), [`Recorder`](https://fluxml.ai/FluxTraining.jl/dev/REFERENCE/FluxTraining.Recorder.html))
- logging modalities
- [x] hyperparameters
- [x] metrics
- logging backends
- [x] TensorBoard ([`TensorBoardBackend`](https://fluxml.ai/FluxTraining.jl/dev/REFERENCE/FluxTraining.TensorBoardBackend.html))
- [x] WandB ([WandB.jl](https://github.com/avik-pal/Wandb.jl))
- [ ] Neptune.ai
- [ ] gradient accumulation

- [ ] metrics and loss functions

Many metrics and loss functions are still missing, see discussion below

### Applications

- computer vision
- image classification
- [x] single-label
- [x] multi-label
- [x] image segmentation
- [x] image keypoint regression
- tabular
- [x] classification
- [x] regression
- [ ] recommender systems
- [ ] natural language processing