lookit / lookit/lookit-api

add video recording S3-Lambda trigger to local environment

Open
#1,290 0 comments 0 reactions 0 assignees View on GitHub
bg-priority Developer
Dominant language
Python
Stars
12
Forks
21
Avg merge
5d 19h
Merged PRs (30d)
5

Description

**TL;DR**:

We need a way to simulate/replicate the new RecordRTC video recording system in our development environment. This is important for developing/testing the experiment runner and the parts of the site that access video recordings (e.g. researcher consent review and response pages, family study history page).

**Narrative**

In the new RecordRTC video recording system, the video files are pushed to S3, which then triggers an AWS Lambda function that calls the Lookit-API. This POST to the Lookit-API is how the video information is actually stored in the Video table in the database.

The problem with this system is that it is not straightforward to re-create in a development environment. During development, we can move the video recordings to AWS S3 and trigger an AWS Lambda function, but it's not clear how we can allow that _external_ AWS Lambda function to communicate with our _local_ lookit instance/database. It's possible to run a local instance of Lambda (with AWS and SAM CLIs), but then we'd run into the same problem with the communication between _external_ AWS S3 storage and _local_ Lambda. Therefore the best solution might be to also run a local instance of S3 storage for videos in the dev environment (which would trigger the local Lambda function and Lookit-API call). We could set this up with new docker containers.

As part of this issue, we should also update the documentation for developers/contributing.

**Acceptance Criteria**

- [ ] Scoping
- [ ] Consider Alternatives
- [ ] Solution allows us to locally use and test changes to the Lambda function
- [ ] Solution allows us to locally develop and test all aspects of the experiment runner and lookit-api
- [ ] Solution is as similar as possible to that used in staging/production (to prevent unexpected problems in those environments)
- [ ] May need to ensure that the solution does not cause problems for local storage space (e.g. regular scheduled clean-up, or Make command to clear video files?)
- [ ] Update documentation for developing with RecordRTC rather than Pipe

Contributor guide

Open the contributing guide

Research direction

No files or tests are named in the issue; begin by tracing the RecordRTC-to-S3-to-Lambda-to-Lookit-API flow and reviewing the existing Pipe development setup. Define and document a local solution that supports Lambda, experiment-runner, and lookit-api testing while remaining close to staging/production and addressing video cleanup.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, python
Domain
backend-api-design, cloud, devops, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.