Lightning-AI / Lightning-AI/pytorch-lightning
Add `on_validation_model_train/eval` to Callback API as well
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Description & Motivation
Today `on_validation_model_train/eval` (similar to on_test_xxx) is only available in `LightningModule`. It would be helpful to open up this interface to callback as well especially as callbacks are much more pluggable. This will be very helpful for implementing ScheduleFree optimizers which has been proved to be very convergence efficient: https://github.com/facebookresearch/schedule_free/tree/main/schedulefree
I'd be happy to draft a PR if this idea is approved.
### Pitch
Add `on_validation_model_train/eval` (and those for test) to callback API
### Alternatives
_No response_
### Additional context
_No response_
cc @lantiga @borda
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the existing on_test_xxx and on_validation_model_train/eval hooks on LightningModule with the callback API. Inspect the callback interfaces and related tests to determine the consistent hook surface; done means validation and test train/eval hooks are exposed to callbacks with coverage for their behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100