Pytorch Lightning Test

I'm trying to find a good pattern for using pytorch lightning. Created some dummy tasks here. Things to consider:

1. Need to tune by searching hyperparameter space.
2. Have more than 1 training run per trial (hyperparameter set). E.g. fold1, fold2.
3. Log summaries with (at least) best val loss achieved and hyperparameter set for all trials.
4. Log epoch-wise training progress (at least train/val loss).
5. Have multiple training stages (e.g. warmup rounds).
6. Possibility to analyze gradients.
7. Run in distributed environment.
8. Evaluate dataloading on every step.

# run trainer
python e5_using_logkey/trainer.py
...
# start tensorboard
tensorboard serve --logdir e5_using_logkey/