Getting corrupted images when reading videos of different sizes

Hi,

DALI seems to sometimes (quite randomly) return invalid images when reading from multiple videos of different sizes (Is it supported?).

When I run the following script:

from tqdm import tqdm
from nvidia import dali
import nvidia.dali.fn as fn
import nvidia.dali.types as types
from nvidia.dali.plugin.pytorch import feed_ndarray
import torch
import torchvision

sintel = ["../../sintel_trailer-1080p-g5.mp4", "../../sintel_trailer-720p-g5.mp4"]
@dali.pipeline_def
def video_pipeline():
    seq = fn.readers.video(device="gpu", filenames=sintel, sequence_length=4, random_shuffle=True, step=5)
    seq = fn.crop(seq, crop_w=256, crop_h=256, crop_pos_x=0, crop_pos_y=0)
    seq = fn.transpose(seq, perm=[0, 3, 1, 2])
    seq = fn.cast(seq, dtype=types.FLOAT) / 255.0
    return seq

pipe = video_pipeline(batch_size=16, num_threads=1, device_id=0)
pipe.build()

torch_batch = torch.zeros((16, 4, 3, 256, 256), device="cuda")
model = torchvision.models.resnet50()
model.cuda()

for i in tqdm(range(99999999)):
    batch = pipe.run()[0]
    feed_ndarray(batch.as_tensor(), torch_batch)
    model(torch_batch[:, 0])
    grid_images = torchvision.utils.make_grid(torch.clamp(torch.cat([torch_batch[i] for i in range(16)], -1), 0, 1),
                                              pad_value=10, nrow=1)

    torchvision.utils.save_image(grid_images, f"{i}.jpg")

I sometimes (quite randomly it seems) get some invalid green images, often before the 10th iteration. The use of a model is optional but seems to make the problem more frequent.

The input videos are encoded in the following way (using sintel trailer 1080p )

ffmpeg -i sintel_trailer-1080p.mp4 -an -g 5 -vf scale=1280x720 sintel_trailer-720p-g5.mp4
ffmpeg -i sintel_trailer-1080p.mp4 -an -g 5 sintel_trailer-1080p-g5.mp4

Here is an example of the output images I get from DALI:

I use DALI 1.4.0 with cuda 10, and it also happens with cuda 11. I see the problem with drivers 465 and 470 but not 440.