[Feature] Add reset_on_start to EarlyStopping
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 726
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 5
Description
Motivation
Currently, the EarlyStopping handler maintains its internal state (counter and best_score) across multiple calls of engine.run(). While this is ideal for resuming training from checkpoints, it creates a friction point for users in interactive environments (e.g., Jupyter Notebooks or Google Colab).
In these settings, a user might:
- Run a training session that triggers early stopping.
- Adjust a hyperparameter in the same notebook.
- Call trainer.run() again using the same engine/handler instance.
Because the state is preserved, the engine immediately terminates because the counter is already at the limit. The user must manually re-instantiate the handler or manually reset the attributes, which is not intuitive for experiment-heavy workflows.
Proposed Solution
I propose adding an optional boolean parameter reset_on_start (defaulting to False) to the EarlyStopping constructor.
If set to True, the handler will attach a listener to Events.STARTED on the provided trainer. This listener calls a internal reset() method to clear self.counter and self.best_score at the start of every run.
The feature is also backwards compatible as the default is set to False which preserve existing behavior for distributed/long-running jobs where state must persist across crashes/resumes.
Example
# The handler will now automatically "refresh" every time trainer.run() is called.
handler = EarlyStopping(
patience=10,
score_function=score_function,
trainer=trainer,
reset_on_start=True
)
evaluator.add_event_handler(Events.COMPLETED, handler)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the EarlyStopping constructor and its existing state handling, then inspect how Events.STARTED listeners are attached. Verify the reset-on-start behavior and the preserved-state default with focused tests covering repeated trainer runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100