How to reduce the memory usage during streaming video inference?

Hi, really appreciate your effort on opening source your marvellous work.

Recently I tried to use SAM3 to perform some segmentation on longer videos (maybe one minute 16fps), however, I've always encountered OOM errors on some samples. The reason is that sometimes it may output too many candidates as detected objects (especially when the concept is a little universal). And I tried streaming inference but I notice that the memory is still incremental during inference (BTW I use the transformers version not the github version).

My questions is as follows:
1. As for the excessive candidates issue, does this mean that it could only be refined by making the prompt more detailed? (e.g. man in white)
2. Why is the memory usage still incremental during streaming inference? Is there any way to avoid/reduce the memory usage growth? (During my practice it could increase nearly 10MB for each frame, which is unendurable...)

Looking forward to your reply to shed light on my doubts, thanks!