Lightning-AI / Lightning-AI/pytorch-lightning

Using a non-named parameter for DataLoader initialization results in an error when using a LightningDataModule

Open
#17,991 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3rd party bug ver: 2.0.x
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

Bug description

When attempting to use the trainer.predict method with a CustomDataModule, an error occurs related to the DataLoader implementation. It seems that multiple values are being passed for the batch_size argument, which results in a TypeError and ultimately terminates the program.
Note: I don't know if it is related but I am using torch geometrics objects (see code below)

What version are you seeing the problem on?

v2.0

How to reproduce the bug
import torch.nn as nn
import torch
import pytorch_lightning as pl
from pytorch_lightning import Trainer
from torch_geometric.data import Data
from torch_geometric.data import InMemoryDataset
from torch_geometric.loader import DataLoader

class CustomData(Data):
    # Very simple random graph data
    def __init__(self, ):
        x = torch.rand(1, 10, 16)  # 10 nodes of shape 16
        edge_index = torch.arange(0, 10).repeat(2, 1)  # Only self edges on each node
        super().__init__(x=x, edge_index=edge_index)


class CustomDataset(InMemoryDataset):
    def __init__(self, num_data=1000, root=None, transform=None, pre_transform=None, pre_filter=None):
        super().__init__(root, transform, pre_transform, pre_filter)
        data_list = [CustomData() for _ in range(num_data)]
        self.data, self.slices = self.collate(data_list)

class CustomDataModule(pl.LightningDataModule):

    def __init__(self):
        super().__init__()
        self.has_setup_fit = False
        self.has_setup_predict = False

    @property
    def batch_size(self):
        return 64

    def setup(self, stage: str):
        if not self.has_setup_fit and stage == 'fit':
            self.dataset = CustomDataset()
            self.has_setup_predict = True
        if not self.has_setup_predict and stage == 'predict':
            self.dataset = CustomDataset()
            self.has_setup_predict = True

    def train_dataloader(self):
        return DataLoader(self.dataset, self.batch_size)

    def val_dataloader(self):
        return DataLoader(self.dataset, self.batch_size)

    def predict_dataloader(self):
        return DataLoader(self.dataset, self.batch_size)


class CustomNeuralNetwork(nn.Module):
    def __init__(self, input_size, layers_size):
        super().__init__()
        layers_list = [nn.Linear(input_size, layers_size[0]), nn.LeakyReLU()]
        for i in range(len(layers_size)-1):
            layers_list.extend([nn.Linear(layers_size[i], layers_size[i+1]), nn.LeakyReLU()])
        self.linears = nn.ModuleList(layers_list)

    def forward(self, x):
        for linear in self.linears:
            x = linear(x)
        return x

class CustomModel(pl.LightningModule):
    def __init__(self):
        super().__init__()
        self.neural_network = CustomNeuralNetwork(16, [64, 32, 16])

    def forward(self, data):
        output = self.neural_network(data.x)
        fake_loss = torch.mean(output**2)
        return fake_loss

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=0.0001)

    def training_step(self, batch, batch_idx, dataloader_idx=0):
        return self(batch)

    def validation_step(self, batch, batch_idx, dataloader_idx=0):
        return self(batch)

    def predict_step(self, batch, batch_idx, dataloader_idx=0):
        return self(batch)

model = CustomModel()
trainer = Trainer(max_epochs=10)
dm = CustomDataModule()
trainer.fit(model, datamodule=dm)
trainer.predict(datamodule=dm)
Error messages and logs

File "C:\Users\Name.Surname\PycharmProjects\ai-developments.venv\lib\site-packages\pytorch_lightning\utilities\data.py", line 133, in _update_dataloader
dataloader = _reinstantiate_wrapped_cls(dataloader, *dl_args, **dl_kwargs)
File "C:\Users\Name.Surname\PycharmProjects\ai-developments.venv\lib\site-packages\lightning_fabric\utilities\data.py", line 280, in _reinstantiate_wrapped_cls
raise MisconfigurationException(message) from e
lightning_fabric.utilities.exceptions.MisconfigurationException: The DataLoader implementation has an error where more than one __init__ argument can be passed to its parent's batch_size=... __init__ argument. This is likely caused by allowing passing both a custom argument that will map to the batch_size argument as well as **kwargs. kwargs should be filtered to make sure they don't contain the batch_size key. This argument was automatically passed to your object by PyTorch Lightning.

Environment
Current environment
#- Lightning Component: Trainer, LightningModule, LightningDataModule
#- PyTorch Lightning Version: 2.0.2
#- PyTorch Version: 1.13.1+cpu
#- Python version: 3.10.9
#- OS: Windows
#- CUDA/cuDNN version: Not used
#- GPU models and configuration: Not used
#- How you installed Lightning(`conda`, `pip`, source): poetry
#- Running environment of LightningApp (e.g. local, cloud): local
More info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the example with the LightningDataModule and positional DataLoader batch_size, then start at pytorch_lightning.utilities.data._update_dataloader and lightning_fabric.utilities.data._reinstantiate_wrapped_cls from the traceback. Check the dataloader re-instantiation path and its tests; done means trainer.predict works without the duplicate batch_size MisconfigurationException.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.