HumanSignal / HumanSignal/label-studio

Metadata Start Time Affects Video Playback Position

Open
#4,714 3 comments 3 reactions 1 assignee Claimed by @makseq View on GitHub
Dominant language
TypeScript
Stars
28.3k
Forks
3.7k
Avg merge
14h
Merged PRs (30d)
15

Description

When using LabelStudio to play videos, I've noticed an issue related to the video's start time defined in its metadata.
`
metadata = FFProbe(video_path)
start = metadata.metadata["start"]
`
This issue arises when a video is cut or trimmed, and its metadata specifies a start time greater than 00:00:00. As a result, LabelStudio starts playing the video from a frame determined by the metadata's start time. This behavior becomes problematic when LabelStudio does not account for the missing frames at the beginning of the video due to the metadata-defined start time.

**Example Scenario:**

* You have a video with 4653 frames.
* The video's metadata specifies a start time greater than 00:00:00 (e.g., 0.066000 seconds).
* The video has a frame rate (FPS) of 29.97.

**Issue Details:**

1. **Start Time Mismatch:** Due to the metadata's start time being greater than 00:00:00, LabelStudio begins playing the video from a frame determined by the start time calculation (e.g., frame 2).
2. **Missing Frames:** LabelStudio plays the video from the calculated frame (frame 2), which means the initial frames (frame 0 and frame 1) are skipped in the playback.
(Frame number computation does not take offset into account and therfore does not start with frame 0. In addition, it cuts off the last frames. Therefore, a workaround of offsetting the annotations does not work)
3. **Misalignment:** Since LabelStudio plays the video starting from frame 2, the frames played correspond to the wrong timing in the video. This misalignment can lead to discrepancies in data annotations and analysis. Even if the offset is added to the frames to correct the mismatches, the frames, that are loaded too much at the beginning, are missing at the end (e.g. 2)

**Screenshots**
First frame at 14. Total Frame Size 504, but because of the offset (start greater than 00:00:00) of 14. Total Frame Size should be 518
![image](https://github.com/HumanSignal/label-studio/assets/127219458/3df9ac0c-1a2e-4c71-bd2d-14d74c49b8a5)

**Environment:**
- Label Studio Version1.8.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.