pytorch / pytorch/vision

Memory leak on GaussianBlur

Open
#6,437 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

module: transforms
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🐛 Describe the bug

Hello. When using num_workers > 0 for dataloader and GaussianBlur BEFORE the resize function in transforms (images in dataset are of different size) a memory leak appears. The larger num_workers used, the more the leak is (I ran out of 128 GB RAM in 300 iterations with batch_size of 32 and num_workers of 16).
To reproduce (you should initialize images with array of filepaths to images):

import torch
import glob
from torchvision import transforms
from PIL import Image
class FramesDataset(torch.utils.data.Dataset):
    def __init__(self, images):
        self.images = images
        self.init_base_transform()

    def __len__(self):
        return len(self.images)

    def init_base_transform(self):
        self.tr_aug = transforms.Compose([transforms.GaussianBlur(7, (1, 5)),
                                          transforms.Resize((256, 256), antialias=True),
                                          transforms.ToTensor(),
                                          transforms.Normalize([0.5]*3, [0.5]*3) ])

    def __getitem__(self, idx):
        img = Image.open(self.images[idx]).convert('RGB')
        out = self.tr_aug(img)
        return out

dataset = TestDataset(images)
dl = torch.utils.data.DataLoader(dataset, batch_size = 16, num_workers = 8, pin_memory = False)
while True:
    for batch in dl:
        pass
Versions

torch: 1.12.0+cu116
torchvision: 0.13.0+cu116
PIL: 9.0.0
Ubuntu: 20.04.4 LTS

cc @vfdev-5 @datumbox

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported transforms.GaussianBlur-before-Resize pipeline and the torch.utils.data.DataLoader reproduction using multiple workers and differently sized images. Compare memory usage across worker counts and isolate whether GaussianBlur, Resize, or their interaction causes the growth; done means the reproduction no longer shows unbounded memory growth and the behavior is covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.