Understanding the output during training

Hey,

Thanks a lot for implementing ENAS in Pytorch! I am able to run ENAS with Penn Treebank dataset. I am trying to understand the output during training. At beginning, the output shows some information about the gradients, for example:

2018-08-29 23:53:50,910:INFO::abs max grad 0.5459082126617432
2018-08-29 23:53:56,523:INFO::abs max grad 0.569364070892334
2018-08-29 23:54:02,990:INFO::abs max grad 0.6024199724197388
2018-08-29 23:54:11,814:INFO::max hidden 16.342870712280273

Then it shows loss and ppl but the epoch number is 0 all the time.
2018-08-29 23:54:17,591:INFO::| epoch 0 | lr 20.00 | raw loss 8.33 | loss 8.33 | ppl 4140.68
2018-08-29 23:54:58,164:INFO::| epoch 0 | lr 20.00 | raw loss 7.29 | loss 7.29 | ppl 1470.41

But after a few epochs, when ppl goes down to 600~, the output changes to show loss and entropy. No ppl is shown.
2018-08-30 00:03:58,605:INFO::| epoch 0 | lr 0.00035 | R 0.13311 | entropy 1.1463 | loss 0.00677
2018-08-30 00:04:10,912:INFO::| epoch 0 | lr 0.00035 | R 0.13406 | entropy 1.1326 | loss 0.01401

Could anyone help me understand the output a little? Why does the output stop showing ppl and why the epoch number is always 0? How to directly monitor the ppl?

Thanks!
Yue

Start by locating the training loop and logging branches used for the Penn Treebank run, then compare the output produced before and after the loss changes described here. Document what the gradient, perplexity, entropy, loss, and epoch fields represent, and identify how perplexity can be monitored directly.