Questions about the TensorRT deployment of the video inference model and video inference flows


I'm currently trying to deploy a video inference model for SAM2 using TensorRT+cpp. Following his idea https://github.com/Aimol-l/OrtInference , split it into four models: image encoder, image decoder, memory encoder, and memory attention. First convert them into onnx files and further generate TensorRT engine files. Currently I have completed the deployment of inference for frame 0 (image encoder + image decoder) modeled after the deployment process of SAM1. However, the inference process for subsequent frames seems to be quite complex, especially the storage and update of obj_ptr and mask_mem. I'm a beginner, are there any detailed explanatory articles/videos for this part of the source code? Or a project on c++ deployment? Much appreciated.
There are a couple of specific questions:
1. after I have completed the inference for frame 0, do the input hints need to be updated when predicting for subsequent frames? (e.g. using the box from the previous frame's inference as the prompt for the new frame?) What should be the input prompt part of the image encoder for frames without prompts?
2. for obj_ptr storage, should it hold the contents of frame 0 (with prompt, conditioned in the paper) + the contents of the 15 most recent frames? If I add a new prompt at frame 20, should it save the contents of frame 20 or the contents of frame 0 (+ the contents of the 15 most recent frames)?
3. https://github.com/Aimol-l/OrtInference adds time coding (the dark green part [7,1,64,64]), I don't know if it exists in the source code, what is its significance?
4. i have some objects that may only exist in certain frames, if i want to reason from a certain frame in the middle of the video, shouldn't the cpp implementation do it by inference backwards + forwards from that frame?