Lightning-AI / Lightning-AI/pytorch-lightning
Support Lightning Logging without Trainer
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
## 🚀 Feature
### Motivation
To ease conversion from pure PyTorch to Lightning, users might start by creating their LightningModule.
However, their code would break if they try to log as the trainer isn't available.
Currently, we have 2 options:
* make `self.log` in the absence of a Trainer
* add support for logging without a trainer.
Here is a pseudo code to explain how we could support it.
The `ResultCollection` object is pretty self contained and is used to store logged values.
```py
class LightningModule:
def __init__(self):
self._lightning_results = ResultCollection
self.training_step = self._training_step_wrapper(self.training_step)
@property
def _results(self):
if getattr(self, "trainer", None) is not None:
return self.trainer._results
return self._lightning_results
def _training_step_wrapper(self, training_step_fn):
def wrapper(self, *args, **kwargs)
self._current_fx = "training_step"
output = training_step_fn(self, *args, **kwargs)
self._current_fx = None
return wrapper
def training_step():
self.log(...)
class Model(LightningModule):
...
model = Model()
for _ in range(epochs):
for batch in datalaoder:
loss = model.training_step(batch, batch_idx)
...
logged_metrics = model.get_logged_metrics()
reduced_metrics = model.get_callback_metrics(epoch=True)
```
Drawback, every LightningModule hooks used for logging should be wrapped to set the `_current_fx` function.
### Pitch
### Alternatives
### Additional context
#### If you enjoy PL, check out our other projects:
- [Metrics](https://github.com/PyTorchLightning/metrics): Machine learning metrics for distributed, scalable PyTorch applications.
- [Flash](https://github.com/PyTorchLightning/lightning-flash): The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning
- [Bolts](https://github.com/PyTorchLightning/lightning-bolts): Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch
- [Lightning Transformers](https://github.com/PyTorchLightning/lightning-transformers): Flexible interface for high performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.
cc @borda @tchaton @justusschock @awaelchli @rohitgr7 @akihironitta @carmocca @edward-io @ananthsub @kamil-kaczmarek @Raalsky @Blaizzy
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading LightningModule, its self.log path, and the ResultCollection object described in the issue. Trace how training_step and other hooks currently obtain trainer state, then define how standalone logging and get_logged_metrics/get_callback_metrics should behave. Done means logging works without a Trainer while preserving trainer-backed behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100