facebookresearch / facebookresearch/sam2

Output video from demo app is missing duration metadata

Open
#539 5 comments 2 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

I'm running into an issue attempting to get the sam2 demo to run properly on a local machine.

I am able to process and download an example file (using the included 01_dog.mp4 as an example), but the output file appears to be missing a lot of metadata, including perhaps most importantly, a duration. The video does play in VLC, but the timeline length shows 00:00 and will not seek to a particular point in time. You can see the same behavior below:

https://github.com/user-attachments/assets/aaf48b4b-1ca6-438c-b1fe-7f6a1f067f72

Looking at the file data in ffmpeg i see some interesting things:

```
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '01_dog_local.mp4':
Metadata:
major_brand : iso4
minor_version : 0
compatible_brands: iso4
Duration: 00:00:00.04, start: 0.000000, bitrate: 728985 kb/s
Stream #0:0[0x1](eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(progressive), 1280x720, 3134 kb/s, 24 fps, 60 tbr, 90k tbn (default)
Metadata:
handler_name : Track created with MP4Box.js
vendor_id : [0][0][0][0]
encoder : avc1 Compressor
Side data:
displaymatrix: rotation of nan degrees
```

this is in contrast to the raw unprocessed example file:

```
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '01_dog.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : My Movie 6
encoder : Lavf61.1.100
keywords : iMovie
Duration: 00:00:09.66, start: 0.000000, bitrate: 1439 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 1429 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
Metadata:
handler_name : Core Media Video
vendor_id : [0][0][0][0]
encoder : Lavc61.3.100 libx264
timecode : 00:00:00:00
Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 2 kb/s (default)
Metadata:
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s
Metadata:
handler_name : Core Media Video
timecode : 00:00:00:00
```

....and the output for the same file when using the sam2 online demo at https://sam2.metademolab.com/demo:
```
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '01_dog_online.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf60.16.100
Duration: 00:00:09.63, start: 0.000000, bitrate: 389 kb/s
Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 386 kb/s, 24 fps, 24 tbr, 12288 tbn (default)
Metadata:
handler_name : Track created with MP4Box.js
vendor_id : [0][0][0][0]
encoder : Lavc60.31.102 libx264
```

The stream data appears to be substantially different - does it look like a corrupted encode? I have tried a different encoder (h264_nvenc) but i'm seeing the same result. Is there an issue with the re-encoding of frames that ffmpeg is doing?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.