Handling batch sizes with fn.experimental.inputs.video

Describe the question.

I want to crop each frame of a video. I currently have a pipeline that takes JPEG images of each frame and crops them, and I am trying to convert it to take the whole video file at once to save time encoding and decoding JPEG files.
For each frame I have a bounding box that defines where I want to crop that video. I am using fn.experimental.inputs.video to parse the video but I am not sure how I can match frames to the proper bounding box. I am basing my code loosely off of the video_decode_remap example in the triton dali backend. My code looks something like this:

def pipeline():
    # Decode video, split into 1 second batches (25 FPS)
    vid = fn.experimental.inputs.video(name="DALI_INPUT_VIDEO", sequence_length=25, device='mixed')
    bbox = dali.fn.external_source(device="cpu", name="DALI_INPUT_BBOX", batch=False)

    # bboxes need to be int32 and of dims [4,1] to work as anchors for slice
    bbox = dali.fn.cast(bbox, dtype=types.INT32)
    bbox = dali.fn.expand_dims(bbox, axes=1)

    bbox_xy = dali.fn.cat(*dali.fn.element_extract(bbox, element_map=[0,1]))
    bbox_wh = dali.fn.cat(*dali.fn.element_extract(bbox, element_map=[2,3]))

    # data, anchor, shape
    crop = dali.fn.slice(images, bbox_xy, bbox_wh)

    return dali.fn.resize(crop, device="gpu", size=(320, 320))

My issue has to do with understanding how batches work. In each batch I am sending a single video file and want to send a single bbox per frame. Calling fn.experimental.inputs.video batches the frames into sequences of length 25, and so I want the bbox variable to be a batch of size 25 of bboxes.

Trying to call the pipeline with DALI_INPUT_VIDEO as a 2 second long video file and DALI_INPUT_BBOX as a dim [50, 4] causes the error:

Exception in CPU stage: [/opt/dali/dali/pipeline/executor/executor.cc:544] Assert on "bsps[0]->NextBatchSize() == bsps[i]->NextBatchSize()" failed: Batch size must be uniform across an iteration
Stacktrace (10 entries):

I also tried to change the sequence_length to 1 but that didn't help. Is there a way to do what I want to do with dali?

Check for duplicates

• I have searched the open bugs/issues and have found no duplicates for this bug report