Lightning-AI / Lightning-AI/pytorch-lightning
Keep User-Defined Order of Callbacks
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
Proposed refactor
Provide a way to disable automatic reordering or completely remove it: https://github.com/Lightning-AI/lightning/blob/7fed7a12c56c6b0e08f0223f68d7d311ea7ac90f/src/pytorch_lightning/trainer/connectors/callback_connector.py#L88
Trainer could take an additional argument reorder_arguments=True.
Motivation
It is not always the case that ModelCheckpoint needs to be the last callback. For example, hyperparameter optimization libraries might externally trigger to kill a Python job based on a feedback before we can checkpoint.
Pitch
Let me use the exact same order of callbacks I've passed to the trainer.
Additional context
Related: #10260
cc @borda @awaelchli @justusschock
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read src/pytorch_lightning/trainer/connectors/callback_connector.py around line 88 and review related issue #10260. Trace how Trainer callbacks are reordered, then determine how the proposed reorder_arguments option should preserve the supplied callback order. Done means the automatic reordering can be disabled or removed without changing unrelated callback behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100