huggingface / huggingface/setfit
EarlyStoppingCallback early_stopping_patience_counter and Trainer.state not reset between hyperparameter search trials
- Dominant language
- Jupyter Notebook
- Stars
- 2.8k
- Forks
- 267
- Avg merge
- 36m
- Merged PRs (30d)
- 5
Description
If using the EarlyStoppingCallback from the transformers package, and running the `Trainer.hyperparameter_search` method, `early_stopping_patience_counter` on the callback instance and `Trainer.state` are not reset between subsequent trials.
This means that when running the second and subsequent trials, the `early_stopping_patience_counter` is greater than the `early_stopping_patience`. This causes `Trainer.control.should_training_stop` to be set to `True` the first time `on_evaluate` is run on the callbacks. The trial is then terminated.
To resolve this issue, I suggest that when calling `Trainer.train(trial=...)` and passing in a trial, `Trainer.callback_hander` should be re-initialised and `Trainer.state` initialised to `TrainerState()`.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.