Missing segmentation mask when tracking multiple objects

I'm trying to track multiple objects in the video sequence,
(_i'm using the magician video in sam **metademolab** site for representation_)

My first click was on 1st frame on the **cloth** for _obj_id=1_, (_**add_new_points_or_box**_)
My second click was on 30th frame on the **hat** for _obj_id=2_

After this, I clicked on the track objects (_**propagate_in_video**_)

Once I run the track objects i observed that at the 30th frame of the video, the segmentation mask for _obj_id=1_ (cloth) is missing.
The same segmentation mask reappear in 31st frame.

Frame 29
Frame 30
Frame 31

Is this a bug or am i missing any configuration setting in my code?

I have a similar code that mimics this whole setup in my local server.

I'm using function _build_sam2_video_predictor_ to load the model, _add_new_points_or_bbox_ to add points, and _propagate_in_video_ to track the objects.

The model i'm using is _sam2_hiera_large.pt_, with cfg file _sam2_hiera_l.yaml_

My desired functionality is to have segmentation mask throughout the video, as if the object is tracked individually.

PS: When there is only one object to be tracked, it is getting tracked without any issue. Or even when the track points for both objects are provided in the same frame then also it is working.

PPS: I looked at the [video_predictor_example.ipynb](https://github.com/facebookresearch/segment-anything-2/blob/main/notebooks/video_predictor_example.ipynb), and [build_sam.py](https://github.com/facebookresearch/segment-anything-2/blob/main/sam2/build_sam.py) (and prolly still looking) for answer to this. Any help is much appreciated.