facebookresearch / facebookresearch/sam2
[Question] Training the Memory Encoder
Open
- Dominant language
- Jupyter Notebook
- Stars
- 19.9k
- Forks
- 2.5k
- PR merge metrics
- No merged PRs in 30d
Description
I read the paper and was a bit confused about how they train the memory encoder. T
hey say they sample 8 frames. Does that mean the memory encoder only ever has 8 frames maximum? What if the object goes out of frame for longer than 8 frames?
Also, since each prediction relies on the memory bank from the previous prediction, does this mean this is a recurrent network architecture, or is there some way to do teacher forcing by feeding in the ground truth?
If someone could help answer these questions or explain how the memory encoder is trainined, that would be super helpful for my understanding!
Contributor guide
Assessment
This issue has not been assessed yet.