Learning Curve Prediction with Bayesian Neural Networks
Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter
Different neural network architectures, hyperparameters and training protocols lead to different performances as a function of time. Human experts routinely inspect the resulting learning curves to quickly terminate runs with poor hyperparameter settings and thereby considerably speed up manual hyperparameter optimization. Exploiting the same information in automatic Bayesian hyperparameter optimization requires a probabilistic model of learning curves across hyperparameter settings. Here, we study the use of Bayesian neural networks for this purpose and improve their performance by a specialized learning curve layer.
Bib (Unofficial source, from dblp)
@inproceedings{DBLP:conf/iclr/KleinFSH17, author = {Aaron Klein and Stefan Falkner and Jost Tobias Springenberg and Frank Hutter}, title = {Learning Curve Prediction with Bayesian Neural Networks}, booktitle = {5th International Conference on Learning Representations, {ICLR} 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings}, publisher = {OpenReview.net}, year = {2017}, url = {https://openreview.net/forum?id=S11KBYclx}, timestamp = {Thu, 25 Jul 2019 14:26:02 +0200}, biburl = {https://dblp.org/rec/conf/iclr/KleinFSH17.bib}, bibsource = {dblp computer science bibliography, https://dblp.org} }