lookit / lookit/lookit-api

Provide video recordings in mp4 format

Open
#1,255 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature
Dominant language
Python
Stars
12
Forks
21
Avg merge
5d 19h
Merged PRs (30d)
5

Description

# TL;DR

WRT issue #1074, our upcoing version of EFP with RecordRTC provides files in `webm` format, which is the most common and widely-supported native file format for audio/video recording in web browsers[^1]. However, researchers have previously received their video files in `mp4` format when downloading from the Lookit/CHS platform, because our Pipe system was set up to automatically transcode files to `mp4` before storing them in S3.

We should consider converting the `webm` video files to `mp4` (either optional or as the default) to maintain the status quo.

We may want to also consider switching to `webm` as the default format, or at least not automatically deleting the `webm` files, since they are the raw recordings. Any transcoded versions will either be equivalent or lossy, and bugs that occur specifically with the conversion step could lead to data loss.

# Narrative

As a researcher, I have been receiving my video files in `mp4` format and would like to continue receivng the same file type.

**Notes**:

This issue is more important for researchers who use the video files as part of another programmatic workflow, such as automatic gaze coding. However I don't actually know what software researchers use for automated workflows and thus whether `webm` support is available alongside `mp4`. This would be helpful to know in order to understand how disruptive this file format change might be.

This issue should not affect researchers who only use the videos as part of a manual workflow (researcher opens each video file), except that the `webm` files may not open in certain applications such as Quicktime.

Also, there are many options available for researchers to convert video files to other formats themselves (though we understand that this is an inconvenience and could be a hurdle in some cases).

# Implementation

There are at least 3 options for implementation:
1. Automatically convert all video files to `mp4` immediately via AWS Lambda when they are saved to S3.
2. Convert video files to `mp4` upon every download request.
3. Use `webm` as the default format for downloads, and convert video files upon a specific request (i.e. add a 'download as mp4' button).

## Option 1: use Lambda

Pros:
- faster download time for the researcher: video files are converted ASAP which means they are immediately available for download
- easier for me to set up: it would involve converting files via the Lambda Python code and storing them in an S3 bucket

Cons:
- creates ~2x the S3 file storage, unless we also delete the `webm` files after conversion (which we probably shouldn't do since the webm file is the original data source)
- makes the Lambda function a little slower and thus a little more expensive

## Options 2/3: use Lookit API

Pros:
- saves storage space: videos are only converted upon request, and can be deleted afterwards.
- makes the Lambda function a little faster/cheaper

Cons:
- researchers would need to wait longer for downloads: we would probably need to email users when their videos are available (at least when file number/size is above some threshold)
- harder for me to set up: it would involve creating a new celery task and/or separate service, adding an email trigger, and adding/modifying video download views

---
[^1]: Confusingly, `video/mp4` is listed as one of the available mime types for video recording:
https://github.com/muaz-khan/RecordRTC#configuration
https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Containers
but this has been shown to still produce `webm` files (at least in some browsers) - see:
https://github.com/streamproc/MediaStreamRecorder/issues/117
https://github.com/collab-project/videojs-record/issues/92
More resources:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types#audio_and_video_types
https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs
https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Containers#webm
https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Containers#mpeg-4_mp4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing Pipe/S3 transcoding behavior and the Lookit API download path, along with the Python Lambda and possible Celery task options described here. Done means selecting and implementing a defined mp4 delivery approach while retaining the original webm recording and covering the resulting download behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
api, backend, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.