Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check compatibility of initial and param_info #97

Merged
merged 8 commits into from
Jan 25, 2024
Merged

Conversation

simonpcouch
Copy link
Contributor

Closes #59. This code just reproduces the portion of dials::encode_unit() methods that picks up on this discrepancy and gives it a more context-specific message.

library(tidymodels)

set.seed(1)

rf_spec <- rand_forest(mode = "regression", mtry = tune())

grid_with_bigger_range <- grid_latin_hypercube(mtry(range = c(1, 16)))

car_folds <- vfold_cv(car_prices, v = 2)

car_wflow <- workflow() %>% 
  add_formula(Price ~ .) %>% 
  add_model(rf_spec)

tune_res_with_bigger_range <- tune_grid(
  car_wflow, 
  resamples = car_folds,
  grid = grid_with_bigger_range
)
parameter_set_with_smaller_range <- parameters(mtry(range = c(1, 5)))

finetune::tune_sim_anneal(
  car_wflow,
  param_info = parameter_set_with_smaller_range,
  resamples = car_folds,
  initial = tune_res_with_bigger_range,
  iter = 2
)
#> Optimizing rmse
#> Initial best: 2570.90000
#> Error in `tune_sim_anneal()`:
#> ! The parameter set used when tuning generating the initial results
#>   isn't compatible with the parameter set supplied as `param_info`.
#> ℹ Possible values of parameters in `param_info` should encompass all values
#>   evaluated in the initial grid.
#>
#> ✖ Optimization stopped prematurely; returning current results.
#> # Tuning results
#> # 2-fold cross-validation 
#> # A tibble: 2 × 5
#>   splits            id    .metrics         .notes           .iter
#>   <list>            <chr> <list>           <list>           <int>
#> 1 <split [402/402]> Fold1 <tibble [6 × 5]> <tibble [0 × 3]>     0
#> 2 <split [402/402]> Fold2 <tibble [6 × 5]> <tibble [0 × 3]>     0

Created on 2024-01-16 with reprex v2.0.2

I've situated this solution so that it only applies to tune_sim_anneal() as tune_bayes() is able to handle that difference just fine.

@@ -144,11 +144,31 @@ random_real_neighbor <- function(current, hist_values, pset, retain = 1,

encode_set_backwards <- function(x, pset, ...) {
pset <- pset[pset$id %in% names(x), ]
new_vals <- purrr::map2(pset$object, x, dials::encode_unit, direction = "backward")
new_vals <- mapply(encode_unit_backwards, pset$object, x,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Transitioned to mapply() so that we don't get purrr's Error in index i, caused by [call we can't control] rethrown error formatting.

R/sim_anneal_helpers.R Outdated Show resolved Hide resolved
R/sim_anneal_helpers.R Show resolved Hide resolved
}

set.seed(1)
expect_snapshot(error = TRUE, transform = scrub_best, {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I didn't know about transform.

* clarify name of helper
* include parameter id in error message
@simonpcouch simonpcouch merged commit e65925b into main Jan 25, 2024
8 checks passed
@simonpcouch simonpcouch deleted the params-59 branch January 25, 2024 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Interaction between initial and param_info args for tune_sim_anneal()
2 participants