ImageFolder dataset: "too many open files" error

Hi,
I am using ImageFolder dataset to train on imagenet.
I repeatedly get "too many open files" OSError after training for several hours.
I suspect the issue comes from [here][1]:

def pil_loader(path):
    # open path as file to avoid ResourceWarning (https://github.com/python-pillow/Pillow/issues/835)
    with open(path, 'rb') as f:
        img = Image.open(f)
        return img.convert('RGB')  # <--

The return is inside with ... clause, thus (I suspect) the file handle f is not closed properly when the function returns.
Shouldn't the function be

def pil_loader(path):
    # open path as file to avoid ResourceWarning (https://github.com/python-pillow/Pillow/issues/835)
    with open(path, 'rb') as f:
        img = Image.open(f)
    return img.convert('RGB')  # <--  not indented

Thanks,
-Shai
[1]: https://github.com/pytorch/vision/blob/master/torchvision/datasets/folder.py#L156

cc @pmeier