You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If using the EarlyStoppingCallback from the transformers package, and running the Trainer.hyperparameter_search method, early_stopping_patience_counter on the callback instance and Trainer.state are not reset between subsequent trials.
This means that when running the second and subsequent trials, the early_stopping_patience_counter is greater than the early_stopping_patience. This causes Trainer.control.should_training_stop to be set to True the first time on_evaluate is run on the callbacks. The trial is then terminated.
To resolve this issue, I suggest that when calling Trainer.train(trial=...) and passing in a trial, Trainer.callback_hander should be re-initialised and Trainer.state initialised to TrainerState().
The text was updated successfully, but these errors were encountered:
Because Trainer.state is not reset between trials, Trainer.state.best_metric has the most recent value from the last trial.
Because EarlyStoppingCallback.early_stopping_patience_counter is not reset between trials, it is incremented each time the callback is evaluated across all runs.
If using the EarlyStoppingCallback from the transformers package, and running the
Trainer.hyperparameter_search
method,early_stopping_patience_counter
on the callback instance andTrainer.state
are not reset between subsequent trials.This means that when running the second and subsequent trials, the
early_stopping_patience_counter
is greater than theearly_stopping_patience
. This causesTrainer.control.should_training_stop
to be set toTrue
the first timeon_evaluate
is run on the callbacks. The trial is then terminated.To resolve this issue, I suggest that when calling
Trainer.train(trial=...)
and passing in a trial,Trainer.callback_hander
should be re-initialised andTrainer.state
initialised toTrainerState()
.The text was updated successfully, but these errors were encountered: