Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support multi-metric in LightGBMTuner and LightGBMTunerCV #143

Open
Alnusjaponica opened this issue Sep 29, 2023 · 2 comments
Open

Support multi-metric in LightGBMTuner and LightGBMTunerCV #143

Alnusjaponica opened this issue Sep 29, 2023 · 2 comments
Labels
contribution-welcome Issue that welcomes contribution. feature Change that does not break compatibility, but affects the public interfaces.

Comments

@Alnusjaponica
Copy link
Contributor

Motivation

Although it is possible to record multiple metric in the original LightGBM (see the official docs), LightGBMTuner and LightGBMTunerCV does not support it. Currently, they just silently utilizes the last metric as the objective here
https://github.com/optuna/optuna/blob/d3fb04fcf41aa1ba6895a22a7b84c55ffdf1cf39/optuna/integration/_lightgbm_tuner/optimize.py#L83-L86 when multiple metrics are given.

Description

  1. The above behavior at least should be noted in the document as mentioned in LightGBMTuner selects last metric and validation set for optimization optuna#3994
  2. It is possible to just record the all given metrics and use the last one as in the past (and as documented) by just stop replacing the given metrics list in _BaseTuner._get_metric_for_objective().

Alternatives (optional)

No response

Additional context (optional)

No response
@Alnusjaponica Alnusjaponica added feature Change that does not break compatibility, but affects the public interfaces. contribution-welcome Issue that welcomes contribution. labels Sep 29, 2023
@Alnusjaponica Alnusjaponica mentioned this issue Sep 30, 2023
Merged
@github-actions GitHub Actions
Copy link

This issue has not seen any recent activity.

@github-actions github-actions bot added the stale Exempt from stale bot labeling. label Jul 25, 2024
@github-actions GitHub Actions
Copy link

This issue was closed automatically because it had not seen any recent activity. If you want to discuss it, you can reopen it freely.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jul 25, 2024
@nzw0301 nzw0301 reopened this Jul 26, 2024
@nzw0301 nzw0301 transferred this issue from optuna/optuna Jul 26, 2024
@nzw0301 nzw0301 removed the stale Exempt from stale bot labeling. label Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
contribution-welcome Issue that welcomes contribution. feature Change that does not break compatibility, but affects the public interfaces.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nzw0301 @Alnusjaponica