Benchmark using pinned CUDA memory in data loaders

Nobody has claimed this yet.

At the moment the data loaders load up images from the dataset, do pre-processing (like normalization), and then convert the images into tensors. Then we copy the data from CPU memory to GPU memory. This can be made more efficient by putting data into page-locked memory and using DMA to the data onto the GPU in async. fashion.

Look into functionality for pinning memory and async and non-blocking data transfers:
- https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader
- https://pytorch.org/docs/stable/notes/cuda.html#use-pinned-memory-buffers

Note: the last time we used this we ran into some PyTorch-internal deadlocks. We need to carefully evaluate this, benchmark it, and figure out if it makes sense to go this route.

Tasks:
- [x] Check out docs for cuda semantics
- [x] Change memory copying behavior
- [ ] Benchmark and test for both training as well as prediction

Start with the existing data-loader memory-copy changes and the linked PyTorch DataLoader and CUDA pinned-memory documentation. Benchmark both training and prediction with pinned memory and asynchronous or non-blocking transfers, while checking for the noted PyTorch-internal deadlocks. Done means the behavior is tested, benchmark results are available, and the approach is shown to be worthwhile or rejected.

Written by the indexing model from the issue text.