Batch input of varying numbers of points

In Sam's forward function, the input parameter is introduced such that when there is a batch input, the dimension of the points can be **BxNx2**. It can be considered that B is the number of objects, and N is the number of points.

When the number of points for multiple objects is different (for example, one object has 1 foreground point coordinate, and another object has 3 foreground point coordinates),

How to batch input when the number of object points is different?

Is it by filling with fake coordinates with a label of **-1**? I noticed that when the label is -1, **self.not_a_point_embed** will be used.

'point_coords': (torch.Tensor) Batched point prompts for
this image, with shape BxNx2. Already transformed to the
input frame of the model.
'point_labels': (torch.Tensor) Batched labels for point prompts,
with shape BxN.
'boxes': (torch.Tensor) Batched box inputs, with shape Bx4.
Already transformed to the input frame of the model.
'mask_inputs': (torch.Tensor) Batched mask inputs to the model,
in the form Bx1xHxW.

Start by reading Sam's forward function and its documented point_coords and point_labels shapes. Check how the existing handling of label -1 and self.not_a_point_embed relates to batching objects with different numbers of points; done would require a confirmed, documented approach for variable-length point inputs.

Written by the indexing model from the issue text.