Lightning-AI / Lightning-AI/pytorch-lightning
Mixing the order of `--config` and `fit` in LightningCLI can cause confusion
Open
@awaelchli is already working on this.
Since Mar 29, 2024.
bug
feature
lightningcli
ver: 2.2.x
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
Bug description
If you launch with
python main.py --config ... fit
instead of
python main.py fit --config ...
Then you end up with cryptic errors such as
No action for key "trainer.accelerator
See report on twitter:
https://x.com/4ndr3aR/status/1772676605837484054?s=20
This is because the LightningCLI parser is built after applying the fit stage is parsed, on only that can match the provided config file. In general, the order matters for jsonargparse for good reasons.
Is there something we can do to improve the error for the user, with a sanity check before parsing begins?
Repro example:
import sys
import torch
from lightning.pytorch import LightningModule
from torch.utils.data import DataLoader, Dataset
from lightning.pytorch.cli import LightningCLI
class RandomDataset(Dataset):
def __init__(self, size, length):
self.len = length
self.data = torch.randn(length, size)
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return self.len
class BoringModel(LightningModule):
def __init__(self):
super().__init__()
self.layer = torch.nn.Linear(32, 2)
def forward(self, x):
return self.layer(x)
def training_step(self, batch, batch_idx):
return self(batch).sum()
def configure_optimizers(self):
return torch.optim.SGD(self.layer.parameters(), lr=0.1)
def train_dataloader(self):
return DataLoader(RandomDataset(32, 64), batch_size=2)
sys.argv = ["bug_report_model.py", "--config", "config.yaml", "fit"]
cli = LightningCLI(BoringModel)
cc @borda @carmocca @mauvilsa
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.