From 9268fbd9caa5014ebc592e6152d91f722c1fa484 Mon Sep 17 00:00:00 2001 From: Jakob Richter Date: Tue, 2 Jan 2018 10:51:07 +0100 Subject: [PATCH] multipoint -> multi-point --- DESCRIPTION | 2 +- LICENSE | 27 ++--------------- R/SMBO.R | 2 +- R/checkStuff.R | 4 +-- R/doc_error_handling.R | 2 +- R/doc_mbo_OptPath.R | 2 +- R/doc_mbo_parallel.R | 2 +- R/getExtras.R | 2 +- R/getSupportedMultipointInfillOptFunctions.R | 4 +-- R/makeMBOControl.R | 4 +-- R/makeProposal.R | 2 +- R/plotExampleRun.R | 4 +-- R/proposePointsHelpers.R | 2 +- R/proposePointsMOIMBO.R | 2 +- R/renderExampleRunPlot.R | 2 +- R/setMBOControlMultiObj.R | 2 +- todo-files/in_depth_intoduction.outdated.Rmd | 4 +-- .../outdated_rmds/in_depth_introduction.Rmd | 30 +++++++++---------- .../supplementary/human_in_the_loop_MBO.Rmd | 2 +- 19 files changed, 39 insertions(+), 62 deletions(-) diff --git a/DESCRIPTION b/DESCRIPTION index 5e4af79e8..e7cd2e40d 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -6,7 +6,7 @@ Description: Flexible and comprehensive R toolbox for model-based optimization conditional parameters. The machine learning toolbox 'mlr' provide dozens of regression learners to model the performance of the target algorithm with respect to the parameter settings. It provides many different infill criteria - to guide the search process. Additional features include multipoint batch + to guide the search process. Additional features include multi-point batch proposal, parallel execution as well as visualization and sophisticated logging mechanisms, which is especially useful for teaching and understanding of algorithm behavior. 'mlrMBO' is implemented in a modular fashion, such that diff --git a/LICENSE b/LICENSE index e543d4651..083b965e2 100644 --- a/LICENSE +++ b/LICENSE @@ -1,25 +1,2 @@ -BSD 2-Clause License - -Copyright (c) 2017, Bernd Bischl -All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: - -* Redistributions of source code must retain the above copyright notice, this - list of conditions and the following disclaimer. - -* Redistributions in binary form must reproduce the above copyright notice, - this list of conditions and the following disclaimer in the documentation - and/or other materials provided with the distribution. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +YEAR: 2018 +COPYRIGHT HOLDER: Bernd Bischl diff --git a/R/SMBO.R b/R/SMBO.R index 65c4df03d..cc9c300e8 100644 --- a/R/SMBO.R +++ b/R/SMBO.R @@ -60,7 +60,7 @@ updateSMBO = function(opt.state, x, y) { assertDataFrame(x) if (nrow(x) > 1 && !is.list(y)) { - stopf("For a multipoint update y has to be a list.") + stopf("For a multi-point update y has to be a list.") } if (!is.list(y)) { y = list(y) diff --git a/R/checkStuff.R b/R/checkStuff.R index ad4471426..810f99929 100644 --- a/R/checkStuff.R +++ b/R/checkStuff.R @@ -112,9 +112,9 @@ checkStuff = function(fun, design, learner, control) { "\nBut this learner does not support prediction of standard errors!")) } if (control$multipoint.method == "cl" && infill.crit.id != "ei") - stopf("Multipoint proposal using constant liar needs the infill criterion 'ei' (expected improvement), but you used '%s'!", infill.crit.id) + stopf("Multi-point proposal using constant liar needs the infill criterion 'ei' (expected improvement), but you used '%s'!", infill.crit.id) if (control$multipoint.method == "cb" && infill.crit.id != "cb") - stopf("Multipoint proposal using parallel cb needs the infill criterion 'cb' (confidence bound), but you used '%s'!", infill.crit.id) + stopf("Multi-point proposal using parallel cb needs the infill criterion 'cb' (confidence bound), but you used '%s'!", infill.crit.id) } } diff --git a/R/doc_error_handling.R b/R/doc_error_handling.R index 6832b2767..9cd429608 100644 --- a/R/doc_error_handling.R +++ b/R/doc_error_handling.R @@ -12,7 +12,7 @@ #' \item{4}{The target function crashes the whole R process.} #' \item{5}{The surrogate machine learning model might crash. #' Kriging quite often can run into numerical problems.} -#' \item{6}{The proposal mechanism - in multipoint or single point mode - produces +#' \item{6}{The proposal mechanism - in multi-point or single point mode - produces #' a point which is either close to another candidate point in the same iteration or #' an already visited point in a previous iteration.} #' \item{7}{The mbo process exits / stops / crashes itself. Maybe because it hit a walltime.} diff --git a/R/doc_mbo_OptPath.R b/R/doc_mbo_OptPath.R index 435569ada..5a07326e8 100644 --- a/R/doc_mbo_OptPath.R +++ b/R/doc_mbo_OptPath.R @@ -17,7 +17,7 @@ #' \item{final\_eval}{If \code{final.evals} is set in \code{\link{makeMBOControl}}: Final evaluations of the proposed solution to reduce noise in y.} #' } #' } -#' \item{parego.weight}{Weight vector sampled for multipoint ParEGO} +#' \item{parego.weight}{Weight vector sampled for multi-point ParEGO} #' \item{...}{Depending on the chosen infill criterion there will be additional columns, e.g. \code{se} and \code{mean} for the Expected Improvement)} #' } #' diff --git a/R/doc_mbo_parallel.R b/R/doc_mbo_parallel.R index 9a25ef801..36dc13e8d 100644 --- a/R/doc_mbo_parallel.R +++ b/R/doc_mbo_parallel.R @@ -16,7 +16,7 @@ #' } #' Details regarding the latter: #' \describe{ -#' \item{single-objective MBO with LCB multipoint}{Parallel optimization of LCBs for the lambda-values.} +#' \item{single-objective MBO with LCB multi-point}{Parallel optimization of LCBs for the lambda-values.} #' \item{Multi-objective MBO with ParEGO}{Parallel optimization of scalarization functions.} #' } #' diff --git a/R/getExtras.R b/R/getExtras.R index c73e49704..1ece1033f 100644 --- a/R/getExtras.R +++ b/R/getExtras.R @@ -28,7 +28,7 @@ getExtras = function(n, prop, train.time, control) { # pregenerate a dummmy "prop" data structure prop = list(crit.vals = matrix(NA_real_, nrow = n, ncol = k), propose.time = NA_real_, errors.model = NA_character_, prop.type = rep("initdesign", n)) # a) no infill crit components for MCO - # b) infill crit is ignored for multipoint moimbo, in which case we don't use any components + # b) infill crit is ignored for multi-point moimbo, in which case we don't use any components if (allow.crit.components) { prop$crit.components = getMBOInfillCritDummyComponents(infill.crit) } diff --git a/R/getSupportedMultipointInfillOptFunctions.R b/R/getSupportedMultipointInfillOptFunctions.R index 2deff87e4..f701e85e3 100644 --- a/R/getSupportedMultipointInfillOptFunctions.R +++ b/R/getSupportedMultipointInfillOptFunctions.R @@ -1,7 +1,7 @@ -#' @title Get names of supported multipoint infill-criteria optimizers. +#' @title Get names of supported multi-point infill-criteria optimizers. #' #' @description -#' Returns all names of supported multipoint infill-criteria optimizers. +#' Returns all names of supported multi-point infill-criteria optimizers. #' #' @return [\code{character}] #' @export diff --git a/R/makeMBOControl.R b/R/makeMBOControl.R index 5e85a4bb0..a3a6c7067 100644 --- a/R/makeMBOControl.R +++ b/R/makeMBOControl.R @@ -80,7 +80,7 @@ #' \dQuote{warn}: The error will be converted to a waring and a random point will be proposed. #' \dQuote{quiet}: Same as “warn” but without the warning. #' Default is: \dQuote{stop}. -#' +#' #' @return [\code{\link{MBOControl}}]. #' @aliases MBOControl #' @family MBOControl @@ -190,7 +190,7 @@ print.MBOControl = function(x, ...) { catf("Infill optimizer : %s", x$infill.opt) catf("Infill optimizer restarts : %i", x$infill.opt.restarts) } else { - catf("Multipoint method : %s", x$multipoint.method) + catf("Multi-point method : %s", x$multipoint.method) } catf("Final point by : %s", x$final.method) } diff --git a/R/makeProposal.R b/R/makeProposal.R index cba82ed66..f898a8b97 100644 --- a/R/makeProposal.R +++ b/R/makeProposal.R @@ -2,7 +2,7 @@ # @param control [MBOcontrol] # @param prop.points [data.frame (nxk)] # columns: each parameter -# rows: each point proposal (usually 1-row, multipoint: more rows) +# rows: each point proposal (usually 1-row, multi-point: more rows) # @param propose.time [numeric (n)] # Vector with time it took for each proposal corresponding to each row (see above) # @param prop.type [character (n)] diff --git a/R/plotExampleRun.R b/R/plotExampleRun.R index a37db473c..928367b5d 100644 --- a/R/plotExampleRun.R +++ b/R/plotExampleRun.R @@ -32,7 +32,7 @@ #' is plotted above and below. #' Default is 1. #' @param single.prop.point.plots [\code{logical(1)}]\cr -#' Parameter for MOI-MBO Multipoint proposal: Should every proposed point +#' Parameter for MOI-MBO Multi-point proposal: Should every proposed point #' be displayed in a single plot - or one plot per Iteration? Default is FALSE #' indicating single plots per proposed points. #' @param xlim [\code{numeric(2)}]\cr @@ -120,7 +120,7 @@ plotExampleRun = function(object, iters, pause = interactive(), single.prop.point.plots = single.prop.point.plots, xlim = xlim, ylim = ylim, point.size = point.size, line.size = line.size, trafo = trafo, colors = colors, gg.objects = gg.objects, ...) if (!any(vlapply(plots, inherits, what = "ggplot"))) { - # in this case we have multi-objective multipoint proposal: list of plots for each proposed point + # in this case we have multi-objective multi-point proposal: list of plots for each proposed point for (jter in seq_along(plots)) { arrangePlots(plots[[jter]], multi.crit) if (pause && !(iter == getLast(iters) && jter == length(plots))) diff --git a/R/proposePointsHelpers.R b/R/proposePointsHelpers.R index 87ef5462b..bdf827630 100644 --- a/R/proposePointsHelpers.R +++ b/R/proposePointsHelpers.R @@ -51,7 +51,7 @@ createSinglePointControls = function(control, crit, crit.pars = NULL) { # perform a deep copy of an opt.path object # so we can store (temporary) stuff in it, without changing the real opt.path -# needed in CL and DIB multipoint +# needed in CL and DIB multi-point deepCopyOptPath = function(op) { op2 = op op2$env = new.env() diff --git a/R/proposePointsMOIMBO.R b/R/proposePointsMOIMBO.R index 53e9f4a44..539596cfb 100644 --- a/R/proposePointsMOIMBO.R +++ b/R/proposePointsMOIMBO.R @@ -48,7 +48,7 @@ proposePointsMOIMBO = function(opt.state, ...) { control = getOptProblemControl(opt.problem) opt.path = getOptStateOptPath(opt.state) - requirePackages("emoa", why = "multipoint InfillOpt MOI-MBO") + requirePackages("emoa", why = "multi-point InfillOpt MOI-MBO") n = control$propose.points objective = control$multipoint.moimbo.objective diff --git a/R/renderExampleRunPlot.R b/R/renderExampleRunPlot.R index d1fc21652..da882d706 100644 --- a/R/renderExampleRunPlot.R +++ b/R/renderExampleRunPlot.R @@ -28,7 +28,7 @@ #' is plotted above and below. #' Default is 1. #' @param single.prop.point.plots [\code{logical(1)}]\cr -#' Parameter for MOI-MBO Multipoint proposal: Should every proposed point +#' Parameter for MOI-MBO Multi-point proposal: Should every proposed point #' be displayed in a single plot - or one plot per Iteration? Default is FALSE #' indicating single plots per proposed points. #' @param xlim [\code{numeric(2)}]\cr diff --git a/R/setMBOControlMultiObj.R b/R/setMBOControlMultiObj.R index 0cb2ae156..4adceaf05 100644 --- a/R/setMBOControlMultiObj.R +++ b/R/setMBOControlMultiObj.R @@ -1,4 +1,4 @@ -#FIXME: briefly explain multipoint proposal for all thre methods +#FIXME: briefly explain multi-point proposal for all three methods #' @title Set multi-objective options. #' @description diff --git a/todo-files/in_depth_intoduction.outdated.Rmd b/todo-files/in_depth_intoduction.outdated.Rmd index 033c19d01..97996bd77 100644 --- a/todo-files/in_depth_intoduction.outdated.Rmd +++ b/todo-files/in_depth_intoduction.outdated.Rmd @@ -140,8 +140,8 @@ _Note:_ You can also create your own stopping condition(s). ### setMBOControlMultiPoint -This extends a MBO control object with options for multipoint proposal. -Multipoint proposal means, that multiple points are proposed and evaluated, which is especially useful for parallel batch evaluation. For a detailed introduction, check the Vignette on parallelization. +This extends a MBO control object with options for multi-point proposal. +Multi-point proposal means, that multiple points are proposed and evaluated, which is especially useful for parallel batch evaluation. For a detailed introduction, check the Vignette on parallelization. ### setMBOControlMultiCrit diff --git a/todo-files/outdated_rmds/in_depth_introduction.Rmd b/todo-files/outdated_rmds/in_depth_introduction.Rmd index 2cf3f6c59..5d7dd5fa4 100644 --- a/todo-files/outdated_rmds/in_depth_introduction.Rmd +++ b/todo-files/outdated_rmds/in_depth_introduction.Rmd @@ -10,7 +10,7 @@ objfun1 = makeBraninFunction() autoplot(objfun1, render.levels = TRUE, show.optimum = TRUE) ``` -The following steps are needed to start a surrogate-based optimization with our package. +The following steps are needed to start a surrogate-based optimization with our package. Each step ends with an R object, which is then passed to ```mbo()```, i.e., to the working horse of **mlrMBO**. 1. define the objective function and its parameters by using the package **smoof** @@ -49,13 +49,13 @@ For more details see `?makeSingleObjectiveFunction`. The second argument of the `mbo()` function - `design` - is the initial design with default setting `NULL`. -An easy (and recommended) way to create an initial design is to use the `generateDesign` function from the **ParamHelpers** package. -If no design is given (i.e. `design = NULL`) a Maximin Latin Hypercube `lhs::maximinLHS` design is used with `n = 4 * getNumberOfParameters(objfun1)` points. -Other possibilities to generate designs are for example `generateGridDesign` and `generateRandomDesign`. +An easy (and recommended) way to create an initial design is to use the `generateDesign` function from the **ParamHelpers** package. +If no design is given (i.e. `design = NULL`) a Maximin Latin Hypercube `lhs::maximinLHS` design is used with `n = 4 * getNumberOfParameters(objfun1)` points. +Other possibilities to generate designs are for example `generateGridDesign` and `generateRandomDesign`. _Note:_ If special designs are desired (e.g., orthogonal designs), they can be given as a `data.frame` but you have to be aware that the output resembles the output of `generateDesign`. -For our objective function `objfun1` we create a slightly larger number of initial points than the default suggests. +For our objective function `objfun1` we create a slightly larger number of initial points than the default suggests. ```{r} n = 5 * getNumberOfParameters(objfun1) design1 = generateDesign(n = n, par.set = getParamSet(objfun1), fun = lhs::randomLHS) @@ -80,13 +80,13 @@ surr.km = makeLearner("regr.km", predict.type = "se", covtype = "matern3_2", con In fact you can use any other regression learner from **mlr** as a surrogate. Depending on the *infill criterion* we will set later it is important that the learner can predict the *uncertainty* (defined by `predict.type = "se"`) alongside the *mean* prediction. -Another popular surrogate is the random forest. +Another popular surrogate is the random forest. It's use is explained in the section for [mixed space optimization](mixed_space_optimization.md). ## MBOControl -The `MBOControl` object controls the optimization process and is created with `makeMBOControl`. +The `MBOControl` object controls the optimization process and is created with `makeMBOControl`. General control arguments can be set when creating it. * `n.objectives`: The number of objectives. 1 implies normal single criteria optimization and is covered in this page. For values >1 see [multi-objective optimization](multi_objective_optimization.md). @@ -103,7 +103,7 @@ It is highly recomended to adjust the settings to suit your optimization problem ### Argument _crit_ -One of the most important questions is to define how the next design points in the sequential loop are chosen. +One of the most important questions is to define how the next design points in the sequential loop are chosen. 5 different infill criteria can be set via the `crit` argument in `setMBOControlInfill`: * `mean`: mean response of the surrogate model @@ -121,14 +121,14 @@ Then points closer then the value of `filter.proposed.points.tol` to an already ### Argument _opt_ -The key idea behind model-based optimization is to substitute the expensive optimization on the black-box with optimization on the surrogate as this is deemed to be cheaper. +The key idea behind model-based optimization is to substitute the expensive optimization on the black-box with optimization on the surrogate as this is deemed to be cheaper. To optimize the infill criterion on the surrogate we also need an optimizer. -The optimum of the infill criterion function gives us the next point that +The optimum of the infill criterion function gives us the next point that Which one to use can be defined with the `opt` argument. The possibilities are: * `focussearch`: A Latin Hypercube design of size `opt.focussearch.points` (default 10000) is sampled in the parameter space (by `randomLHS`) and the design point with the best prediction of the infill criterion is determined. Then, the parameter space is shrunk around this point. This step is repeated `opt.focussearch.maxit` (default 5) times and the best observed point is passed back. -* `cmaes`: The optimal point is found with a covariance matrix adapting evolutionary strategy from the **cmaes** package. If the strategy fails, a random point is generated and a warning is given. Further control arguments can be provided in `opt.cmaes.control` as a list. +* `cmaes`: The optimal point is found with a covariance matrix adapting evolutionary strategy from the **cmaes** package. If the strategy fails, a random point is generated and a warning is given. Further control arguments can be provided in `opt.cmaes.control` as a list. * `ea`: Use an evolutionary multiobjective optimization algorithm from the package **emoa** to determine the best point. The population size mu can be set by `opt.ea.mu` (default value is 10). (mu+1) means that in each population only one child is generated using crossover und mutation operators. The parameters `eta` and `p` of the latter two operators can be adjusted via the attributes `opt.ea.sbx.eta`, `opt.ea.sbx.p`,`opt.ea.pm.eta` and `opt.ea.pm.p`. The default number of EA iterations is 500 and can be changed by `opt.ea.maxit` attribute. * `nsga2`: Use the non-dominated sorting genetic algorithm from the package **nsga2R** to determine the best point. This algorithm should be used for [multi objective optimization](multi_objective_optimization.md). @@ -141,7 +141,7 @@ _Note:_ Only the `focussearch` optimizer is suitable for for categorical paramet ## setMBOControlTermination With this control function different criteria to stop the fitting process can be specified. You can set multiple different criteria and the first one that is met will terminate the optimization process. -You can set: +You can set: * `iters`: The maximum number of iterations * `time.budget`: A maximum running time in seconds @@ -152,11 +152,11 @@ _Note:_ You can also easily create your own stopping condition(s). ## setMBOControlMultiPoint -This extends a MBO control object with options for multipoint proposal. Multipoint proposal means, that multiple points are proposed and evaluated, which is especially useful for parallel batch evaluation. For a detailed introduction, check the [multi-point tutorial](multipoint.md). +This extends a MBO control object with options for multi-point proposal. Multi-point proposal means, that multiple points are proposed and evaluated, which is especially useful for parallel batch evaluation. For a detailed introduction, check the [multi-point tutorial](multipoint.md). ### Argument: _method_ -Define the method used for multipoint proposals, currently 3 different methods are supported: +Define the method used for multi-point proposals, currently 3 different methods are supported: * `cb`: Proposes multiple points by optimizing the confidence bound criterion `propose.points` times with different lambda values. Generally this works the same way as for the single point case, i.e. specify `infill.opt`. The lambdas are drawn from an exp(1)-distribution. * `multicrit`: Use a evolutionary multicriteria optimization. This is a (mu+1) type evolutionary algorithm and runs for `multicrit.maxit` generations. The population size is set to `propose.points`. @@ -164,7 +164,7 @@ Define the method used for multipoint proposals, currently 3 different methods a ## setMBOControlMultiFid -Add multi-fidelity options to the `MBOControl` control object. This is useful when certain parameters increase the performance as well as the calculation cost. The idea is to combine the optimization of fast fitting low-fidelity models and more accurate but expensive high-fidelity models. The parameter on which the fidelity depends on is specified as `param` and the order of the values to train the learner with in `lvls`. The costs for the different levels can be specified directly or estimated by a model based on the execution time of the currently evaluated points. +Add multi-fidelity options to the `MBOControl` control object. This is useful when certain parameters increase the performance as well as the calculation cost. The idea is to combine the optimization of fast fitting low-fidelity models and more accurate but expensive high-fidelity models. The parameter on which the fidelity depends on is specified as `param` and the order of the values to train the learner with in `lvls`. The costs for the different levels can be specified directly or estimated by a model based on the execution time of the currently evaluated points. ## setMBOControlMultiCrit diff --git a/vignettes/supplementary/human_in_the_loop_MBO.Rmd b/vignettes/supplementary/human_in_the_loop_MBO.Rmd index 86c4c7d84..b830c3000 100644 --- a/vignettes/supplementary/human_in_the_loop_MBO.Rmd +++ b/vignettes/supplementary/human_in_the_loop_MBO.Rmd @@ -152,7 +152,7 @@ updateSMBO(opt.state, x = prop$prop.points, y = y) ## Proposal of multiple points -Using multipoint-MBO you can also obtain multiple suggestions at each call of `proposePoints()`. +Using Multi-Point MBO you can also obtain multiple suggestions at each call of `proposePoints()`. ```{r multipoint} ctrl = makeMBOControl(propose.points = 4)