AutoML MulticlassClassificationExperiment minimizes LogLossReduction — MultiClassMetricManager.IsMaximize returns false for it #7839

Defect

src/Microsoft.ML.AutoML/AutoMLExperiment/IMetricManager.cs, MultiClassMetricManager.IsMaximize:

MulticlassClassificationMetric.LogLoss => false,
MulticlassClassificationMetric.LogLossReduction => false,   // should be true

Log-loss reduction is 1 - LogLoss / LogLoss(prior) — higher is better (1 = perfect, 0 = no better than the class prior, negative = worse). With IsMaximize == false the trial loss is the metric itself and the experiment's best run is the minimum, so an experiment optimizing it returns the model that learned least.

Reproduction (Microsoft.ML 5.0.1 / Microsoft.ML.AutoML 0.23.1, osx-arm64)

150 rows, 3 classes, one feature that separates them almost perfectly (a = class + U(0,1)), MulticlassExperimentSettings.OptimizingMetric = LogLossReduction, 10 s budget:

• trials: FastTreeOva 0.0016 → 0.4176, FastForestOva 0.9606, LbfgsMaximumEntropy 0.6961, … then the tuner settles on SdcaMaximumEntropy at −0.599 for the rest of the budget
• returned best: macro accuracy 0.333, log loss 1.0986 (= ln 3), log-loss reduction 0.0
• same data with OptimizingMetric = LogLoss: macro accuracy 1.0, log-loss reduction 1.0

Expected

LogLossReduction => true.

Related observation (not a defect claim)

TopKAccuracy as an optimizing metric: the experiment evaluates with the default topKPredictionCount = 0, for which MulticlassClassificationMetrics.TopKAccuracy is 0 for every model (measured: micro accuracy 1, top-k accuracy 0). MulticlassExperimentSettings has no setting to pass K, so optimizing it compares constants.