Skip to content

Commit

Permalink
Merge pull request #714 from mlr-org/submission
Browse files Browse the repository at this point in the history
Submission
  • Loading branch information
RaphaelS1 authored Jul 4, 2023
2 parents 09ec526 + 51e01f7 commit 8fd0801
Show file tree
Hide file tree
Showing 178 changed files with 34,396 additions and 16 deletions.
13 changes: 6 additions & 7 deletions book/chapters/chapter1/introduction_and_overview.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,7 @@ If you want more detail about any of the tasks used in this book or links to all

#### Reproducibility {.unnumbered .unlisted}

This book was built under Ubuntu 22.04.2 LTS on an x86_64-pc-linux-gnu (64-bit) platform using R 4.3.1.
At the start of each chapter we run `set.seed(123)` and use `ref_pkg("renv")` to manage package versions, you can find our lockfile at `r link("https://github.com/mlr-org/mlr3book/blob/main/book/renv.lock")`.
At the start of each chapter we run `set.seed(123)` and use `r ref_pkg("renv")` to manage package versions, you can find our lockfile at `r link("https://github.com/mlr-org/mlr3book/blob/main/book/renv.lock")`.


## mlr3book Code Style {#styleguide}
Expand Down Expand Up @@ -254,29 +253,29 @@ As both are core to `mlr3` we *briefly* introduce both packages for beginners; i
If you have experience with any (class) object-oriented programming then R6 should feel familiar.
We focus on the parts of R6 that you need to know to use `mlr3`.

`r index('Objects', "objects", parent = "R6", aside = TRUE, lower = FALSE)` are created by constructing an instance of an `r ref("R6::R6Class")` variable using the `$new()` initialization method.
*Objects* are created by constructing an instance of an `r ref("R6::R6Class")` variable using the `$new()` initialization method.
For example, say we have implemented a class called `Foo`, then `foo = Foo$new(bar = 1)` would create a new object of class `Foo` and set the `bar` argument of the constructor to the value `1`.
In practice, we implement a lot of sugar functionality (@sec-mlr3-utilities) in `mlr3` that make construction and access a bit more convenient.

Some `R6` objects may have mutable states that are encapsulated in their `r index('fields', parent = "R6", aside = TRUE, lower = FALSE)`, which can be accessed through the dollar, `$`, operator.
Some `R6` objects may have mutable states that are encapsulated in their *fields*, which can be accessed through the dollar, `$`, operator.
Continuing the previous example, we can access the `bar` value in the `foo` object by using `foo$bar` or we could give it a new value, e.g. `foo$bar = 2`.
These fields can also be 'active bindings', which perform additional computations when referenced or modified.

In addition to fields, `r index('methods', parent = "R6", aside = TRUE, lower = FALSE)` allow users to inspect the object's state, retrieve information, or perform an action that changes the internal state of the object.
In addition to fields, *methods* allow users to inspect the object's state, retrieve information, or perform an action that changes the internal state of the object.
For example, in `mlr3`, the `$train()` method of a learner changes the internal state of the learner by building and storing a model.
Methods that modify the internal state of an object often return the object itself.
Other methods may return a new R6 object.
In both cases, it is possible to 'chain' methods by calling one immediately after the other using the `$`-operator; this is similar to the `%>%`-operator used in `tidyverse` packages.
For example, `Foo$bar()$hello_world()` would run the `$bar()` method of the object `Foo` and then the `$hello_world()` method of the object returned by `$bar()` (which may be `Foo` itself).

Fields and methods can be public or private\index{R6!methods!private}\index{R6!methods!public}\index{R6!fields!private}\index{R6!fields!public}.
Fields and methods can be public or private.
The public fields and methods define the API to interact with the object.
In `mlr3`, you can safely ignore private methods unless you are looking to extend our universe by adding a new class (@sec-technical).

Finally, `R6` objects are `environments`, and as such have reference semantics.
This means that, for example, `foo2 = foo` does not create a new variable called `foo2` that is a copy of `foo`.
Instead, it creates a variable called `foo2` that references `foo`, and so setting `foo$bar = 3` will also change `foo2$bar` to `3` and vice versa.
To copy an object, use the `$clone(deep = TRUE)` method, so to copy `foo`: `foo2 = foo$clone(deep = TRUE)`[`$clone()`]{.aside}\index{R6!\texttt{\$clone()}}.
To copy an object, use the `$clone(deep = TRUE)` method, so to copy `foo`: `foo2 = foo$clone(deep = TRUE)`[`$clone()`]{.aside}.

For a longer introduction, we recommend the `R6` vignettes found at `r link("https://r6.r-lib.org/")`; more detail can be found in `r link("https://adv-r.hadley.nz/r6.html")`.

Expand Down
2 changes: 1 addition & 1 deletion book/chapters/chapter11/large-scale_benchmarking.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ For the rest of this section, we will look at how to use `batchtools` and `r mlr
```{r large_benchmarking-026, echo = FALSE}
#| label: fig-hpc
#| fig-cap: "Illustration of an HPC cluster architecture."
#| fig-alt: "Figure shows a flow diagram of objects. Far left is a laptop with an arrow connecting it to an object that says 'Head Node - Scheduler', the arrow has the text 'SSH'. The scheduler has a bidirectional arrow with text 'Submit' to an object that says 'Queue' that has an arrow to an object 'Computing Nodes'. The scheduler also has an arrow to 'File System' which has a double arrow connecting it to/from the 'Computing Nodes' object with text 'Data'."
#| fig-alt: "Flow diagram of objects. Left is a laptop with an arrow to an object that says 'Head Node - Scheduler', the arrow has text 'SSH'. The scheduler has a bidirectional arrow with text 'Submit' to 'Queue' that has an arrow to 'Computing Nodes'. The scheduler also has an arrow to 'File System' which has a double arrow connecting it to/from the 'Computing Nodes' object with text 'Data'."
include_multi_graphics("mlr3book_figures-32")
```

Expand Down
2 changes: 1 addition & 1 deletion book/chapters/chapter12/model_interpretation.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -698,7 +698,7 @@ If the features can be meaningfully grouped, grouped versions of methods, e.g. t

## Exercises

The following exercises are based on predictions of the value of soccer players based on their characteristics in the FIFA video game series. They use the 2020 data available in DALEX (`r ref("DALEX::fifa")`). Solve them with either `iml` or `DALEX`.
The following exercises are based on predictions of the value of soccer players based on their characteristics in the FIFA video game series. They use the 2020 `fifa` data available in DALEX. Solve them with either `iml` or `DALEX`.

1. Prepare an `mlr3` regression task for the `fifa` data. Select only features describing the age and skills of soccer players. Train a predictive model of your own choice on this task, to predict the value of a soccer player.
2. Use the permutation importance method to calculate feature importance ranking. Which feature is the most important? Do you find the results surprising?
Expand Down
3 changes: 1 addition & 2 deletions book/chapters/chapter3/evaluation_and_benchmarking.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -623,8 +623,7 @@ Instead, several normalized measures can be derived (@fig-confusion):
#| echo: false
#| label: fig-confusion
#| fig-cap: "Binary confusion matrix of ground truth class vs. predicted class."
#| fig-alt: "Binary confusion matrix of ground truth class vs. predicted class."
#| fig-alt: "Representation of a confusion matrix with entries 'TP' when both 'True class y' is '+'' and 'Predicted Class yhat' is '+', 'TN' when both are '-', 'FP' when True is '-' and Predicted is '+' and finally 'FN' when True is '+' but Predicted is '-'. In the margins is 'PPV = TP/(TP+FP)', 'NPV = TN/(FN+TN)', 'ACC=(TP+TN)/(TP+FP+FN+TN)', 'TNR=TN/(FP+TN)', 'TPR=TP/(TP+FN)'."
include_multi_graphics("confusion_matrix")
```

Expand Down
6 changes: 3 additions & 3 deletions book/chapters/chapter4/hyperparameter_optimization.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ There are multiple `r ref("Tuner", aside = TRUE)` classes in `mlr3tuning`, which
| Grid Search | `tnr("grid_search")` | `r mlr3tuning` |
| Bayesian Optimization | `tnr("mbo")` | `r mlr3mbo` |
| CMA-ES | `tnr("cmaes")` | `r ref_pkg("adagio")` |
| Iterative Racing | `tnr("irace")` | `r ref_pkg("irace")` |
| Iterated Racing | `tnr("irace")` | `r ref_pkg("irace")` |
| Hyperband | `tnr("hyperband")` | `r mlr3hyperband` |
| Generalized Simulated Annealing | `tnr("gensa")` | `r ref_pkg("GenSA")` |
| Nonlinear Optimization | `tnr("nloptr")` | `r ref_pkg("nloptr")` |
Expand All @@ -174,7 +174,7 @@ Due to their simplicity, both grid search and random search can handle mixed sea

#### Adaptive algorithms {.unnumbered .unlisted}

Adaptive algorithms learn from previously evaluated configurations to find good configurations quickly, examples in `r mlr3` include Bayesian optimization (also called model-based optimization), Covariance Matrix Adaptation Evolution Strategy (CMA-ES), `r index('Iterative Racing')`, and Hyperband.
Adaptive algorithms learn from previously evaluated configurations to find good configurations quickly, examples in `r mlr3` include Bayesian optimization (also called model-based optimization), Covariance Matrix Adaptation Evolution Strategy (CMA-ES), `r index('Iterated Racing')`, and Hyperband.

`r index('Bayesian optimization', lower = FALSE)` [e.g., @Snoek2012] describes a family of iterative optimization algorithms that use a surrogate model to approximate the unknown function that is to be optimized -- in HPO this would be the mapping from a hyperparameter configuration to the estimated generalization performance. If a suitable surrogate model is chosen, e.g. a random forest,
Bayesian optimization can be quite flexible and even handle mixed and hierarchical search spaces. Bayesian optimization is discussed in full detail in @sec-bayesian-optimization.
Expand All @@ -186,7 +186,7 @@ Further evolutionary strategies are available in `mlr3` via the `r ref_pkg("mies


Racing algorithms work by iteratively discarding configurations that show poor performance, as determined by statistical tests.
Iterative Racing [@lopez2016] starts by 'racing' down an initial population of randomly sampled configurations from a parameterized density and then uses the surviving configurations of the race to stochastically update the density of the subsequent race to focus on promising regions of the search space, and so on.
Iterated Racing [@lopez2016] starts by 'racing' down an initial population of randomly sampled configurations from a parameterized density and then uses the surviving configurations of the race to stochastically update the density of the subsequent race to focus on promising regions of the search space, and so on.


Multi-fidelity HPO is an adaptive method that leverages the predictive power of computationally cheap lower fidelity evaluations (i.e., poorer quality predictions such as those arising from neural networks with a small number of epochs) to improve the overall optimization efficiency.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ graph$plot(horizontal = TRUE)
```{r 05-pipelines-multicol-5-evalT, fig.width = 8, eval = TRUE, echo = FALSE}
#| label: fig-pipelines-selectnop
#| fig-cap: Visualization of our `Graph` where features are split into two paths, features that start with 'bill' are scaled and the rest are untransformed.
#| fig-alt: 'Six boxes where first is "<INPUT>" which points to "s_bill -> scale" and "s_notbill -> nop", then both "scale" and "nop" point to "featureunion -> <OUTPUT>".'
#| fig-alt: 'Seven boxes where first is "<INPUT>" which points to "s_bill -> scale" and "s_notbill -> nop", then both "scale" and "nop" point to "featureunion -> <OUTPUT>".'
graph = gunion(list(
po_select_bill %>>% po("scale"),
po_select_not_bill %>>% po("nop")
Expand Down Expand Up @@ -481,7 +481,7 @@ tsk_mnist = as_task(otsk_mnist)$
```

`po("branch")` is initialized either with the number of branches or with a `character`-vector indicating the names of the branches, the latter makes the `selection` hyperparameter (discussed below) more readable.
Below we create three branches: do nothing (`po("nop")`), apply PCA (`po("pca")`), remove constant features (`po("removeconstants")`) then apply the `r index('Yeo-Johnson')` transform (`po("yeojohnson")`).
Below we create three branches: do nothing (`po("nop")`), apply PCA (`po("pca")`), remove constant features (`po("removeconstants")`) then apply the `r index('Yeo-Johnson', lower = FALSE)` transform (`po("yeojohnson")`).
It is important to use `po("unbranch")` (with the same arguments as `"branch"`) to ensure that the outputs are merged into one result object.

```{r 05-pipelines-non-sequential-003, eval = FALSE}
Expand Down
Binary file not shown.
Loading

0 comments on commit 8fd0801

Please sign in to comment.