google-deepmind / google-deepmind/video_comp
Reproduction Details
- Dominant language
- No language data
- Stars
- 7
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Dear Authors,
I have a few questions while reproducing the finetuning process and hope you can help clarify.
1. The number of training samples for each type of negative example is not uniform. How should the hierarchical pairwise preference loss mentioned in the paper be applied? For example, one video has A > B > C, while another has A > B > C > D. This seems difficult to implement with batch training.
2. For the seg_mismatch type, according to the article's meaning, the video is cropped from "query_video/start_time" to "query_video/end_time" and then the similarity is calculated with "positive_text" and "negative_text" respectively. So, for the other negative texts corresponding to this video, (such as temp_reorder, action_replace), should their similarity be calculated with the original video or the cropped video?
3. How is the Multi-Disrupt trainingset implemented? Are there any key parameters or strategies to note? In the training process, are the negative samples constructed online (dynamically) or pre-generated offline (fixed)?
Thanks~
Contributor guide
Assessment
This issue has not been assessed yet.