HumanSignal / HumanSignal/label-studio
Metadata Start Time Affects Video Playback Position
- Dominant language
- TypeScript
- Stars
- 28.3k
- Forks
- 3.7k
- Avg merge
- 14h
- Merged PRs (30d)
- 15
Description
When using LabelStudio to play videos, I've noticed an issue related to the video's start time defined in its metadata.
`
metadata = FFProbe(video_path)
start = metadata.metadata["start"]
`
This issue arises when a video is cut or trimmed, and its metadata specifies a start time greater than 00:00:00. As a result, LabelStudio starts playing the video from a frame determined by the metadata's start time. This behavior becomes problematic when LabelStudio does not account for the missing frames at the beginning of the video due to the metadata-defined start time.
**Example Scenario:**
* You have a video with 4653 frames.
* The video's metadata specifies a start time greater than 00:00:00 (e.g., 0.066000 seconds).
* The video has a frame rate (FPS) of 29.97.
**Issue Details:**
1. **Start Time Mismatch:** Due to the metadata's start time being greater than 00:00:00, LabelStudio begins playing the video from a frame determined by the start time calculation (e.g., frame 2).
2. **Missing Frames:** LabelStudio plays the video from the calculated frame (frame 2), which means the initial frames (frame 0 and frame 1) are skipped in the playback.
(Frame number computation does not take offset into account and therfore does not start with frame 0. In addition, it cuts off the last frames. Therefore, a workaround of offsetting the annotations does not work)
3. **Misalignment:** Since LabelStudio plays the video starting from frame 2, the frames played correspond to the wrong timing in the video. This misalignment can lead to discrepancies in data annotations and analysis. Even if the offset is added to the frames to correct the mismatches, the frames, that are loaded too much at the beginning, are missing at the end (e.g. 2)
**Screenshots**
First frame at 14. Total Frame Size 504, but because of the offset (start greater than 00:00:00) of 14. Total Frame Size should be 518

**Environment:**
- Label Studio Version1.8.0
Contributor guide
Assessment
This issue has not been assessed yet.