Notes for Future Implementations

For future optimization regarding data transfer when using DataParallel https://discuss.pytorch.org/t/how-to-prefetch-data-when-processing-with-gpu/548/19

Determinism https://github.com/pytorch/pytorch/issues/18412 https://github.com/pytorch/pytorch/issues/15359

Tensors must be created on the correct device, as noted in 1.1.0 https://github.com/pytorch/pytorch/releases

we can now use modelparallel https://pytorch.org/tutorials/intermediate/model_parallel_tutorial.html