multi-gpus, reproduce loss #15
Description
I have implmented a multi-gpus version.
1>But I am not sure how to evaluate reproduce to be successful for the search process.What's the correct evaluation accuracy in the training process?
2>And I find the loss is large in the search training process, like this issue mentioned. Is that a normal value for loss? I found it's related with "runtime_lambda_val", and 0.02 is the correct setting for it?
3>And "base_learning_rate" is 0.016, it seem to be small for training from scratch, though it's enlarged by " FLAGS.base_learning_rate * (FLAGS.train_batch_size / 256.0)", and is it the correct setting?