How to manage empty items in a batch?

Describe the question.

Thanks in advance for your help.

I'm running into an issue in a pipeline with ~11 operators. During processing, some processing steps may become irrelevant for certain items in the batch. For these empty batch items, processing should be skipped for any subsequent operators.

Currently, it seems to be required to implement workarounds for this by setting the returned Tensors to contain signal values. For some operators, I can get away by returning a Tensor (i.e. a torch Tensor from a fn.torch_python_function, or a cupy.ndarray from a fn.python_function) with a shape with first dimension set to 0, for instance (0, 640, 640, 3). But this does not always work (some operators raise exceptions), and it has been required to return bogus arrays containing -1 values in some cases. In custom operators, it is then required to test for these signal values, and to skip processing and return empty values for these empty batch items.

Below a code snippet to (hopefully) clarify:

def postprocess(previous_step_output_batch, ...):
    postprocess_output_batch = []
    for previous_step_output_sample in previous_step_output_batch:
        if (... condition that will produce a valid output sample ...):
            ....
            postprocess_output_sample = ...
        else:
            # Create sample to represent an empty batch item
            postprocess_output_sample = torch.ones(0, 640, 640, 3) * -1
            # In some scenarios (other functions, other shapes), it's required to return a shape with first dimension > 0
            # other_output_sample = torch.ones(1, 6) * -1
        postprocess_output_batch.append(postprocess_output_sample.to("cuda"))
    return postprocess_output_batch
...
def create_pipeline(...):
    ...
    postprocess_output_batch = dalitorch.fn.torch_python_function(
        previous_step_output_batch
        , function=lambda input1: postprocess(input1, ...)
        , batch_processing=True
        , device="gpu"
    )

Note that this implementation uses batch_processing=True. Would this be different/improved if using batch_processing=False? (i.e. does DALI then check for empty/None batch items?)

In general, what is the correct approach to deal with empty batch items?

Check for duplicates

• I have searched the open bugs/issues and have found no duplicates for this bug report