Lightning-AI / Lightning-AI/pytorch-lightning
check_val_every_n_epoch bug with list of dataloaders
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
## 🐛 Bug
If you have a list of dataloaders (used in a continual learning setting) and `check_val_every_n_epoch` in the trainer is not equal to 1 then `validation_step()` won't return the index of the dataloader currently being used.
### To Reproduce
Dataloader for validation:
```
def val_dataloader(self):
return [DataLoader(ds, batch_size=self.batch_size, shuffle=True , num_workers=self.num_workers) for ds in self.test_datasets[:self.curr_index+1]]
```
Validation step:
```
def validation_step(self, val_batch, batch_idx, dataloader_idx=None):
if dataloader_idx:
print("data loader index provided")
```
Run this in a training loop with `check_val_every_n_epoch=1` and compare to `check_val_every_n_epoch=5`. The latter will always have `dataloader_idx=None`.
### Expected behavior
The dataloader_idx is always correctly provided independently of how often the validation is called.
### Environment
- PyTorch Lightning Version (e.g., 1.5.0): 1.5.4
- PyTorch Version (e.g., 1.10): 1.10.0 py3.8_cuda10.2_cudnn7.6.5_0
- Python version (e.g., 3.9): 3.8.3
- OS (e.g., Linux): Linux
- CUDA/cuDNN version: 10.1
- GPU models and configuration: v100
- How you installed PyTorch (`conda`, `pip`, source): conda
### Additional context
Thanks.
cc @carmocca @justusschock @ananthsub @ninginthecloud @rohitgr7
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.