Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster evaluation for MPM2 #149

Merged
merged 8 commits into from
Oct 20, 2023

Conversation

schaepermeier
Copy link
Contributor

This PR adds support for extracting all instance-specific information for MPM2 problems from the original Python environment, and subsequently evaluating the problems within R instead of Python. This behavior is controlled by an additional parameter evaluation.env, which per default is set to the faster R.
It can be significantly faster in multi-objective optimization settings as the Python interface always only stores the configuration of the latest MPM2 function called, leading to expensive reconfigurations for each batch of evaluations.
Even in single-objective settings it is somewhat faster (about 20% for me).

In multi-objective settings it can easily be orders of magnitude faster when the MPM2 instance generation algorithm takes comparatively long to configure the problem. For example, on my machine:

> env = "Python"
> 
> fn1 = makeMPM2Function(16, 2, "random", 8, evaluation.env = env)
> fn2 = makeMPM2Function(16, 2, "random", 9, evaluation.env = env)
> 
> system.time({
+   sapply(1:1e2, function(i) {
+     fn1(c(0.5, 0.5))
+     fn2(c(0.5, 0.5))
+   })
+ })
   user  system elapsed 
  1.239   0.002   0.613 
> library(smoof)
> 
> env = "R"
> 
> fn1 = makeMPM2Function(16, 2, "random", 8, evaluation.env = env)
> fn2 = makeMPM2Function(16, 2, "random", 9, evaluation.env = env)
> 
> system.time({
+   sapply(1:1e2, function(i) {
+     fn1(c(0.5, 0.5))
+     fn2(c(0.5, 0.5))
+   })
+ })
   user  system elapsed 
  0.018   0.000   0.011 
> 

Tests to check matching outputs are also provided. Let me know if you need me to change anything!

Cheers,
Lennart

Copy link
Owner

@jakobbossek jakobbossek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@jakobbossek
Copy link
Owner

There are some conflicts with the NEWS files most likely because I just merged your PR on the extended biobj. BBOB suite. Please fix ASAP.

@schaepermeier
Copy link
Contributor Author

That should fix it :-)

R/sof.mpm2.R Outdated
@@ -12,6 +12,12 @@
#' of elliptically shaped peaks.
#' @param peak.shape [\code{character(1)}]\cr
#' Shape of peak(s). Possible values are \dQuote{ellipse} and \dQuote{sphere}.
#' @param evaluation.env [\code{character(1)}]\cr
#' Evaluation environment after the function was created. Possible values are
#' \dQuote{R} (default) and \dQuote{Python}. The original generation of the
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, the case-sensitivity here is actually not user-friendly. Should we allow arbitrary string? E.g., r or R or pYthon or PyThOn?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I made it case-insensitive now

@@ -4,6 +4,8 @@ smoof 1.6.0.4
* Fixed bug in getLoggedValues when logging of x-values was set to FALSE in addLoggingWrapper
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General comment: given your nice contribution I suggest to add yourself to the list of contributors in DESCRIPTION 🥳

Copy link
Owner

@jakobbossek jakobbossek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@jakobbossek
Copy link
Owner

Thanks Lennart for your super-fast responses! Merging 😎

@jakobbossek jakobbossek merged commit 9e31848 into jakobbossek:master Oct 20, 2023
3 of 5 checks passed
@schaepermeier schaepermeier deleted the mpm2-fast-evaluation branch October 20, 2023 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants