Lightning-AI / Lightning-AI/pytorch-lightning
LightningModule.configure_callbacks overrides Trainer callbacks
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Outline & Motivation
In [TorchGeo](https://github.com/microsoft/torchgeo), we provide `LightningModules` for common tasks, like classification, regression, etc. We also provide default callbacks using `LightningModule.configure_callbacks` as suggested in #18480. However, it seems that these default callbacks override any user-specific callbacks provided to the `Trainer`. According to [the documentation](https://lightning.ai/docs/pytorch/stable/common/lightning_module.html#configure-callbacks), this actually seems to be the intended behavior:
> the list or a callback returned here will be merged with the list of callbacks passed to the Trainer’s `callbacks` argument. If a callback returned here has the same type as one or several callbacks already present in the Trainer’s callbacks list, it will take priority and replace them.
I propose reversing this behavior such that Trainer `callbacks` override `LightningModule.configure_callbacks` when both share the same type.
### Pitch
Users should be able to override callbacks hard-coded in the `LightningModule` with whatever they pass to the `Trainer`. Otherwise the only way to change something like the model checkpoint frequency is to subclass the `LightningModule`, which is unideal.
Alternatively, if there's another way to provide model-specific default callbacks, please let me know.
### Additional context
@robmarkcole and @roybenhayun reported several issues to us that stemmed from this behavior. If we can't solve this, we may have to remove all default callbacks.
@calebrob6
cc @borda @carmocca @justusschock @awaelchli
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the LightningModule.configure_callbacks entry point and the Trainer callbacks behavior described in the linked documentation. Reproduce how same-type callbacks are merged and replaced, then review the discussion and related reports before deciding the intended precedence. Done means the documented behavior and tests establish whether Trainer callbacks override module defaults, or clarify an alternative for model-specific defaults.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100