dandi / dandi/dandi-cli

"sense" video files as for the quality of their compression, seekability, ...

Open
#1,575 1 comment 0 reactions 0 assignees View on GitHub
ember
Dominant language
Python
Stars
28
Forks
37
Avg merge
1d 17h
Merged PRs (30d)
9

Description

Came up in a meetup with @talmo re their experiences in SLEAP working with video recordings of behavioral data. In their experience, many, if not most, videos coming from audio/video recording hardware would be either uncompressed or badly compressed, resulting in those recordings taking LOTS of space (placing the aspect of number of files aside for now). For that reason in https://sleap.ai/help.html#does-my-data-need-to-be-in-a-particular-format they provide recommended settings for ffmpeg to see such files compressed:

```
ffmpeg -y -i "input.mp4" -c:v libx264 -pix_fmt yuv420p -preset superfast -crf 23 "output.mp4"
```

Similarly, for ReproNim's reprostim project, where we capture audio/video of presented to MRI experiment stimuli using cheap magewell capture card, and on a small underpowered hardware, with @vmdocua we specify default settings in https://github.com/ReproNim/reprostim/blob/master/src/reprostim-capture/videocapture/config.yaml#L105 for ffmpeg video component as

```
-c:v libx264 -flush_packets 1 -preset ultrafast -crf 18 -tune zerolatency -b:v 8M -maxrate 8M -bufsize 16M -vf setpts=PTS-STARTPTS
```

ATM.

edited: In DANDI we already have 44 dandisets with videos

```shell
dandi@drogon:/mnt/backup/dandi/dandisets$ for f in 0*; do find $f -regextype posix-extended -iregex '.*\.(mp4|avi|wmv|mov|flv|mkv)'; done >| /tmp/videos.txt
dandi@drogon:/mnt/backup/dandi/dandisets$ grep -v '\.git' /tmp/videos.txt | sed -e 's,/.*,,g' | sort | uniq -c | sort -nr | nl
1 2483 000559
2 899 000409
3 794 000779
4 495 000540
5 459 000951
6 400 000780
7 360 000793
8 360 000792
9 319 000782
10 319 000781
11 264 000831
12 264 000830
13 256 000867
14 256 000866
15 115 000231
16 105 000833
17 105 000832
18 82 000568
19 64 001084
20 40 000807
21 40 000806
22 40 000805
23 40 000804
24 40 000803
25 40 000802
26 40 000801
27 40 000800
28 32 000689
29 19 000624
30 18 000863
31 18 000862
32 16 001172
33 9 001259
34 9 000727
35 9 000576
36 8 001190
37 8 000718
38 3 001195
39 2 001180
40 1 001265
41 1 000892
42 1 000691
43 1 000360
44 1 000167
```

I think it would be

- useful to provide some guidance at the time of validation/upload to ensure that videos uploaded to archive are appropriately compressed and we are not blowing up the size of the archive with loosely or uncompressed videos. Yet to figure out usable procedures for that but could be pretty much taking a sample video and recompressing with "recommended" settings and seeing how well it compresses
- validate that videos are seekable (ref: https://github.com/janclemenslab/napari-video/issues/3 per @talmo [comment below](https://github.com/dandi/dandi-cli/issues/1575#issuecomment-2653942958))
- reviewing current videos in the archive and issue such recommendations?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.