DAMO-NLP-SG / DAMO-NLP-SG/VideoLLaMA3
How to prepare fine-tuning data for the temporal grounding task?
- Dominant language
- Jupyter Notebook
- Stars
- 1.2k
- Forks
- 89
- PR merge metrics
- No merged PRs in 30d
Description
I noticed that you use the YouCook2 dataset for temporal grounding. When I prepare the data in this format, I modify the stage4_2b.sh script and replace the data_path with the YouCook2 jsonl file for fine-tuning. However, I encountered an issue where the process fails to start due to the lack of data_dict["conversations"]. It seems that the YouCook2 dataset doesn't have it. Could you please clarify how the data format for temporal grounding is handled on your side, especially how the temporal information is incorporated into the labels?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with stage4_2b.sh and the YouCook2 JSONL input, then trace where data_dict["conversations"] is required. Done means documenting the accepted fine-tuning format and explaining how temporal information is represented in the labels, with a working preparation path if the repository provides one.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- shell
- Domain
- data, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100