You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks a lot for implementing ENAS in Pytorch! I am able to run ENAS with Penn Treebank dataset. I am trying to understand the output during training. At beginning, the output shows some information about the gradients, for example:
2018-08-29 23:53:50,910:INFO::abs max grad 0.5459082126617432
2018-08-29 23:53:56,523:INFO::abs max grad 0.569364070892334
2018-08-29 23:54:02,990:INFO::abs max grad 0.6024199724197388
2018-08-29 23:54:11,814:INFO::max hidden 16.342870712280273
Then it shows loss and ppl but the epoch number is 0 all the time.
2018-08-29 23:54:17,591:INFO::| epoch 0 | lr 20.00 | raw loss 8.33 | loss 8.33 | ppl 4140.68
2018-08-29 23:54:58,164:INFO::| epoch 0 | lr 20.00 | raw loss 7.29 | loss 7.29 | ppl 1470.41
But after a few epochs, when ppl goes down to 600~, the output changes to show loss and entropy. No ppl is shown.
2018-08-30 00:03:58,605:INFO::| epoch 0 | lr 0.00035 | R 0.13311 | entropy 1.1463 | loss 0.00677
2018-08-30 00:04:10,912:INFO::| epoch 0 | lr 0.00035 | R 0.13406 | entropy 1.1326 | loss 0.01401
Could anyone help me understand the output a little? Why does the output stop showing ppl and why the epoch number is always 0? How to directly monitor the ppl?
Thanks!
Yue
The text was updated successfully, but these errors were encountered:
Hey,
Thanks a lot for implementing ENAS in Pytorch! I am able to run ENAS with Penn Treebank dataset. I am trying to understand the output during training. At beginning, the output shows some information about the gradients, for example:
2018-08-29 23:53:50,910:INFO::abs max grad 0.5459082126617432
2018-08-29 23:53:56,523:INFO::abs max grad 0.569364070892334
2018-08-29 23:54:02,990:INFO::abs max grad 0.6024199724197388
2018-08-29 23:54:11,814:INFO::max hidden 16.342870712280273
Then it shows loss and ppl but the epoch number is 0 all the time.
2018-08-29 23:54:17,591:INFO::| epoch 0 | lr 20.00 | raw loss 8.33 | loss 8.33 | ppl 4140.68
2018-08-29 23:54:58,164:INFO::| epoch 0 | lr 20.00 | raw loss 7.29 | loss 7.29 | ppl 1470.41
But after a few epochs, when ppl goes down to 600~, the output changes to show loss and entropy. No ppl is shown.
2018-08-30 00:03:58,605:INFO::| epoch 0 | lr 0.00035 | R 0.13311 | entropy 1.1463 | loss 0.00677
2018-08-30 00:04:10,912:INFO::| epoch 0 | lr 0.00035 | R 0.13406 | entropy 1.1326 | loss 0.01401
Could anyone help me understand the output a little? Why does the output stop showing ppl and why the epoch number is always 0? How to directly monitor the ppl?
Thanks!
Yue
The text was updated successfully, but these errors were encountered: