Skip to content

Commit

Permalink
multipoint -> multi-point
Browse files Browse the repository at this point in the history
  • Loading branch information
jakob-r committed Jan 2, 2018
1 parent 71651f4 commit 9268fbd
Show file tree
Hide file tree
Showing 19 changed files with 39 additions and 62 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Description: Flexible and comprehensive R toolbox for model-based optimization
conditional parameters. The machine learning toolbox 'mlr' provide dozens
of regression learners to model the performance of the target algorithm with
respect to the parameter settings. It provides many different infill criteria
to guide the search process. Additional features include multipoint batch
to guide the search process. Additional features include multi-point batch
proposal, parallel execution as well as visualization and sophisticated
logging mechanisms, which is especially useful for teaching and understanding
of algorithm behavior. 'mlrMBO' is implemented in a modular fashion, such that
Expand Down
27 changes: 2 additions & 25 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,25 +1,2 @@
BSD 2-Clause License

Copyright (c) 2017, Bernd Bischl
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
YEAR: 2018
COPYRIGHT HOLDER: Bernd Bischl
2 changes: 1 addition & 1 deletion R/SMBO.R
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ updateSMBO = function(opt.state, x, y) {

assertDataFrame(x)
if (nrow(x) > 1 && !is.list(y)) {
stopf("For a multipoint update y has to be a list.")
stopf("For a multi-point update y has to be a list.")
}
if (!is.list(y)) {
y = list(y)
Expand Down
4 changes: 2 additions & 2 deletions R/checkStuff.R
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,9 @@ checkStuff = function(fun, design, learner, control) {
"\nBut this learner does not support prediction of standard errors!"))
}
if (control$multipoint.method == "cl" && infill.crit.id != "ei")
stopf("Multipoint proposal using constant liar needs the infill criterion 'ei' (expected improvement), but you used '%s'!", infill.crit.id)
stopf("Multi-point proposal using constant liar needs the infill criterion 'ei' (expected improvement), but you used '%s'!", infill.crit.id)
if (control$multipoint.method == "cb" && infill.crit.id != "cb")
stopf("Multipoint proposal using parallel cb needs the infill criterion 'cb' (confidence bound), but you used '%s'!", infill.crit.id)
stopf("Multi-point proposal using parallel cb needs the infill criterion 'cb' (confidence bound), but you used '%s'!", infill.crit.id)
}
}

Expand Down
2 changes: 1 addition & 1 deletion R/doc_error_handling.R
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
#' \item{4}{The target function crashes the whole R process.}
#' \item{5}{The surrogate machine learning model might crash.
#' Kriging quite often can run into numerical problems.}
#' \item{6}{The proposal mechanism - in multipoint or single point mode - produces
#' \item{6}{The proposal mechanism - in multi-point or single point mode - produces
#' a point which is either close to another candidate point in the same iteration or
#' an already visited point in a previous iteration.}
#' \item{7}{The mbo process exits / stops / crashes itself. Maybe because it hit a walltime.}
Expand Down
2 changes: 1 addition & 1 deletion R/doc_mbo_OptPath.R
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
#' \item{final\_eval}{If \code{final.evals} is set in \code{\link{makeMBOControl}}: Final evaluations of the proposed solution to reduce noise in y.}
#' }
#' }
#' \item{parego.weight}{Weight vector sampled for multipoint ParEGO}
#' \item{parego.weight}{Weight vector sampled for multi-point ParEGO}
#' \item{...}{Depending on the chosen infill criterion there will be additional columns, e.g. \code{se} and \code{mean} for the Expected Improvement)}
#' }
#'
Expand Down
2 changes: 1 addition & 1 deletion R/doc_mbo_parallel.R
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#' }
#' Details regarding the latter:
#' \describe{
#' \item{single-objective MBO with LCB multipoint}{Parallel optimization of LCBs for the lambda-values.}
#' \item{single-objective MBO with LCB multi-point}{Parallel optimization of LCBs for the lambda-values.}
#' \item{Multi-objective MBO with ParEGO}{Parallel optimization of scalarization functions.}
#' }
#'
Expand Down
2 changes: 1 addition & 1 deletion R/getExtras.R
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ getExtras = function(n, prop, train.time, control) {
# pregenerate a dummmy "prop" data structure
prop = list(crit.vals = matrix(NA_real_, nrow = n, ncol = k), propose.time = NA_real_, errors.model = NA_character_, prop.type = rep("initdesign", n))
# a) no infill crit components for MCO
# b) infill crit is ignored for multipoint moimbo, in which case we don't use any components
# b) infill crit is ignored for multi-point moimbo, in which case we don't use any components
if (allow.crit.components) {
prop$crit.components = getMBOInfillCritDummyComponents(infill.crit)
}
Expand Down
4 changes: 2 additions & 2 deletions R/getSupportedMultipointInfillOptFunctions.R
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#' @title Get names of supported multipoint infill-criteria optimizers.
#' @title Get names of supported multi-point infill-criteria optimizers.
#'
#' @description
#' Returns all names of supported multipoint infill-criteria optimizers.
#' Returns all names of supported multi-point infill-criteria optimizers.
#'
#' @return [\code{character}]
#' @export
Expand Down
4 changes: 2 additions & 2 deletions R/makeMBOControl.R
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@
#' \dQuote{warn}: The error will be converted to a waring and a random point will be proposed.
#' \dQuote{quiet}: Same as “warn” but without the warning.
#' Default is: \dQuote{stop}.
#'
#'
#' @return [\code{\link{MBOControl}}].
#' @aliases MBOControl
#' @family MBOControl
Expand Down Expand Up @@ -190,7 +190,7 @@ print.MBOControl = function(x, ...) {
catf("Infill optimizer : %s", x$infill.opt)
catf("Infill optimizer restarts : %i", x$infill.opt.restarts)
} else {
catf("Multipoint method : %s", x$multipoint.method)
catf("Multi-point method : %s", x$multipoint.method)
}
catf("Final point by : %s", x$final.method)
}
Expand Down
2 changes: 1 addition & 1 deletion R/makeProposal.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# @param control [MBOcontrol]
# @param prop.points [data.frame (nxk)]
# columns: each parameter
# rows: each point proposal (usually 1-row, multipoint: more rows)
# rows: each point proposal (usually 1-row, multi-point: more rows)
# @param propose.time [numeric (n)]
# Vector with time it took for each proposal corresponding to each row (see above)
# @param prop.type [character (n)]
Expand Down
4 changes: 2 additions & 2 deletions R/plotExampleRun.R
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
#' is plotted above and below.
#' Default is 1.
#' @param single.prop.point.plots [\code{logical(1)}]\cr
#' Parameter for MOI-MBO Multipoint proposal: Should every proposed point
#' Parameter for MOI-MBO Multi-point proposal: Should every proposed point
#' be displayed in a single plot - or one plot per Iteration? Default is FALSE
#' indicating single plots per proposed points.
#' @param xlim [\code{numeric(2)}]\cr
Expand Down Expand Up @@ -120,7 +120,7 @@ plotExampleRun = function(object, iters, pause = interactive(),
single.prop.point.plots = single.prop.point.plots, xlim = xlim, ylim = ylim,
point.size = point.size, line.size = line.size, trafo = trafo, colors = colors, gg.objects = gg.objects, ...)
if (!any(vlapply(plots, inherits, what = "ggplot"))) {
# in this case we have multi-objective multipoint proposal: list of plots for each proposed point
# in this case we have multi-objective multi-point proposal: list of plots for each proposed point
for (jter in seq_along(plots)) {
arrangePlots(plots[[jter]], multi.crit)
if (pause && !(iter == getLast(iters) && jter == length(plots)))
Expand Down
2 changes: 1 addition & 1 deletion R/proposePointsHelpers.R
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ createSinglePointControls = function(control, crit, crit.pars = NULL) {

# perform a deep copy of an opt.path object
# so we can store (temporary) stuff in it, without changing the real opt.path
# needed in CL and DIB multipoint
# needed in CL and DIB multi-point
deepCopyOptPath = function(op) {
op2 = op
op2$env = new.env()
Expand Down
2 changes: 1 addition & 1 deletion R/proposePointsMOIMBO.R
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ proposePointsMOIMBO = function(opt.state, ...) {
control = getOptProblemControl(opt.problem)
opt.path = getOptStateOptPath(opt.state)

requirePackages("emoa", why = "multipoint InfillOpt MOI-MBO")
requirePackages("emoa", why = "multi-point InfillOpt MOI-MBO")

n = control$propose.points
objective = control$multipoint.moimbo.objective
Expand Down
2 changes: 1 addition & 1 deletion R/renderExampleRunPlot.R
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
#' is plotted above and below.
#' Default is 1.
#' @param single.prop.point.plots [\code{logical(1)}]\cr
#' Parameter for MOI-MBO Multipoint proposal: Should every proposed point
#' Parameter for MOI-MBO Multi-point proposal: Should every proposed point
#' be displayed in a single plot - or one plot per Iteration? Default is FALSE
#' indicating single plots per proposed points.
#' @param xlim [\code{numeric(2)}]\cr
Expand Down
2 changes: 1 addition & 1 deletion R/setMBOControlMultiObj.R
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#FIXME: briefly explain multipoint proposal for all thre methods
#FIXME: briefly explain multi-point proposal for all three methods

#' @title Set multi-objective options.
#' @description
Expand Down
4 changes: 2 additions & 2 deletions todo-files/in_depth_intoduction.outdated.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,8 @@ _Note:_ You can also create your own stopping condition(s).

### setMBOControlMultiPoint

This extends a MBO control object with options for multipoint proposal.
Multipoint proposal means, that multiple points are proposed and evaluated, which is especially useful for parallel batch evaluation. For a detailed introduction, check the Vignette on parallelization.
This extends a MBO control object with options for multi-point proposal.
Multi-point proposal means, that multiple points are proposed and evaluated, which is especially useful for parallel batch evaluation. For a detailed introduction, check the Vignette on parallelization.

### setMBOControlMultiCrit

Expand Down
30 changes: 15 additions & 15 deletions todo-files/outdated_rmds/in_depth_introduction.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ objfun1 = makeBraninFunction()
autoplot(objfun1, render.levels = TRUE, show.optimum = TRUE)
```

The following steps are needed to start a surrogate-based optimization with our package.
The following steps are needed to start a surrogate-based optimization with our package.
Each step ends with an R object, which is then passed to ```mbo()```, i.e., to the working horse of **mlrMBO**.

1. define the objective function and its parameters by using the package **smoof**
Expand Down Expand Up @@ -49,13 +49,13 @@ For more details see `?makeSingleObjectiveFunction`.

The second argument of the `mbo()` function - `design` - is the initial design with default setting `NULL`.

An easy (and recommended) way to create an initial design is to use the `generateDesign` function from the **ParamHelpers** package.
If no design is given (i.e. `design = NULL`) a Maximin Latin Hypercube `lhs::maximinLHS` design is used with `n = 4 * getNumberOfParameters(objfun1)` points.
Other possibilities to generate designs are for example `generateGridDesign` and `generateRandomDesign`.
An easy (and recommended) way to create an initial design is to use the `generateDesign` function from the **ParamHelpers** package.
If no design is given (i.e. `design = NULL`) a Maximin Latin Hypercube `lhs::maximinLHS` design is used with `n = 4 * getNumberOfParameters(objfun1)` points.
Other possibilities to generate designs are for example `generateGridDesign` and `generateRandomDesign`.

_Note:_ If special designs are desired (e.g., orthogonal designs), they can be given as a `data.frame` but you have to be aware that the output resembles the output of `generateDesign`.

For our objective function `objfun1` we create a slightly larger number of initial points than the default suggests.
For our objective function `objfun1` we create a slightly larger number of initial points than the default suggests.
```{r}
n = 5 * getNumberOfParameters(objfun1)
design1 = generateDesign(n = n, par.set = getParamSet(objfun1), fun = lhs::randomLHS)
Expand All @@ -80,13 +80,13 @@ surr.km = makeLearner("regr.km", predict.type = "se", covtype = "matern3_2", con

In fact you can use any other regression learner from **mlr** as a surrogate.
Depending on the *infill criterion* we will set later it is important that the learner can predict the *uncertainty* (defined by `predict.type = "se"`) alongside the *mean* prediction.
Another popular surrogate is the random forest.
Another popular surrogate is the random forest.
It's use is explained in the section for [mixed space optimization](mixed_space_optimization.md).


## MBOControl

The `MBOControl` object controls the optimization process and is created with `makeMBOControl`.
The `MBOControl` object controls the optimization process and is created with `makeMBOControl`.
General control arguments can be set when creating it.

* `n.objectives`: The number of objectives. 1 implies normal single criteria optimization and is covered in this page. For values >1 see [multi-objective optimization](multi_objective_optimization.md).
Expand All @@ -103,7 +103,7 @@ It is highly recomended to adjust the settings to suit your optimization problem

### Argument _crit_

One of the most important questions is to define how the next design points in the sequential loop are chosen.
One of the most important questions is to define how the next design points in the sequential loop are chosen.
5 different infill criteria can be set via the `crit` argument in `setMBOControlInfill`:

* `mean`: mean response of the surrogate model
Expand All @@ -121,14 +121,14 @@ Then points closer then the value of `filter.proposed.points.tol` to an already

### Argument _opt_

The key idea behind model-based optimization is to substitute the expensive optimization on the black-box with optimization on the surrogate as this is deemed to be cheaper.
The key idea behind model-based optimization is to substitute the expensive optimization on the black-box with optimization on the surrogate as this is deemed to be cheaper.
To optimize the infill criterion on the surrogate we also need an optimizer.
The optimum of the infill criterion function gives us the next point that
The optimum of the infill criterion function gives us the next point that
Which one to use can be defined with the `opt` argument.
The possibilities are:

* `focussearch`: A Latin Hypercube design of size `opt.focussearch.points` (default 10000) is sampled in the parameter space (by `randomLHS`) and the design point with the best prediction of the infill criterion is determined. Then, the parameter space is shrunk around this point. This step is repeated `opt.focussearch.maxit` (default 5) times and the best observed point is passed back.
* `cmaes`: The optimal point is found with a covariance matrix adapting evolutionary strategy from the **cmaes** package. If the strategy fails, a random point is generated and a warning is given. Further control arguments can be provided in `opt.cmaes.control` as a list.
* `cmaes`: The optimal point is found with a covariance matrix adapting evolutionary strategy from the **cmaes** package. If the strategy fails, a random point is generated and a warning is given. Further control arguments can be provided in `opt.cmaes.control` as a list.
* `ea`: Use an evolutionary multiobjective optimization algorithm from the package **emoa** to determine the best point. The population size mu can be set by `opt.ea.mu` (default value is 10). (mu+1) means that in each population only one child is generated using crossover und mutation operators. The parameters `eta` and `p` of the latter two operators can be adjusted via the attributes `opt.ea.sbx.eta`, `opt.ea.sbx.p`,`opt.ea.pm.eta` and `opt.ea.pm.p`. The default number of EA iterations is 500 and can be changed by `opt.ea.maxit` attribute.
* `nsga2`: Use the non-dominated sorting genetic algorithm from the package **nsga2R** to determine the best point. This algorithm should be used for [multi objective optimization](multi_objective_optimization.md).

Expand All @@ -141,7 +141,7 @@ _Note:_ Only the `focussearch` optimizer is suitable for for categorical paramet
## setMBOControlTermination

With this control function different criteria to stop the fitting process can be specified. You can set multiple different criteria and the first one that is met will terminate the optimization process.
You can set:
You can set:

* `iters`: The maximum number of iterations
* `time.budget`: A maximum running time in seconds
Expand All @@ -152,19 +152,19 @@ _Note:_ You can also easily create your own stopping condition(s).

## setMBOControlMultiPoint

This extends a MBO control object with options for multipoint proposal. Multipoint proposal means, that multiple points are proposed and evaluated, which is especially useful for parallel batch evaluation. For a detailed introduction, check the [multi-point tutorial](multipoint.md).
This extends a MBO control object with options for multi-point proposal. Multi-point proposal means, that multiple points are proposed and evaluated, which is especially useful for parallel batch evaluation. For a detailed introduction, check the [multi-point tutorial](multipoint.md).

### Argument: _method_

Define the method used for multipoint proposals, currently 3 different methods are supported:
Define the method used for multi-point proposals, currently 3 different methods are supported:

* `cb`: Proposes multiple points by optimizing the confidence bound criterion `propose.points` times with different lambda values. Generally this works the same way as for the single point case, i.e. specify `infill.opt`. The lambdas are drawn from an exp(1)-distribution.
* `multicrit`: Use a evolutionary multicriteria optimization. This is a (mu+1) type evolutionary algorithm and runs for `multicrit.maxit` generations. The population size is set to `propose.points`.
* `cl`: Proposes points by the constant liar strategy, which only makes sense if the confidence bound criterion is used as an infill criterion. In the first step the surrugate model is fitted based on the real data and the best point is calculated accordingly. Then, the function value of the best point is simply guessed by the worst seen function evaluation. This "lie"" is used to update the model in order to propose subsequent point. The procedure is applied until the number of points has reached `propose.points`.

## setMBOControlMultiFid

Add multi-fidelity options to the `MBOControl` control object. This is useful when certain parameters increase the performance as well as the calculation cost. The idea is to combine the optimization of fast fitting low-fidelity models and more accurate but expensive high-fidelity models. The parameter on which the fidelity depends on is specified as `param` and the order of the values to train the learner with in `lvls`. The costs for the different levels can be specified directly or estimated by a model based on the execution time of the currently evaluated points.
Add multi-fidelity options to the `MBOControl` control object. This is useful when certain parameters increase the performance as well as the calculation cost. The idea is to combine the optimization of fast fitting low-fidelity models and more accurate but expensive high-fidelity models. The parameter on which the fidelity depends on is specified as `param` and the order of the values to train the learner with in `lvls`. The costs for the different levels can be specified directly or estimated by a model based on the execution time of the currently evaluated points.

## setMBOControlMultiCrit

Expand Down
2 changes: 1 addition & 1 deletion vignettes/supplementary/human_in_the_loop_MBO.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ updateSMBO(opt.state, x = prop$prop.points, y = y)

## Proposal of multiple points

Using multipoint-MBO you can also obtain multiple suggestions at each call of `proposePoints()`.
Using Multi-Point MBO you can also obtain multiple suggestions at each call of `proposePoints()`.

```{r multipoint}
ctrl = makeMBOControl(propose.points = 4)
Expand Down

0 comments on commit 9268fbd

Please sign in to comment.