add video recording S3-Lambda trigger to local environment
- Dominant language
- Python
- Stars
- 12
- Forks
- 21
- Avg merge
- 5d 19h
- Merged PRs (30d)
- 5
Description
**TL;DR**:
We need a way to simulate/replicate the new RecordRTC video recording system in our development environment. This is important for developing/testing the experiment runner and the parts of the site that access video recordings (e.g. researcher consent review and response pages, family study history page).
**Narrative**
In the new RecordRTC video recording system, the video files are pushed to S3, which then triggers an AWS Lambda function that calls the Lookit-API. This POST to the Lookit-API is how the video information is actually stored in the Video table in the database.
The problem with this system is that it is not straightforward to re-create in a development environment. During development, we can move the video recordings to AWS S3 and trigger an AWS Lambda function, but it's not clear how we can allow that _external_ AWS Lambda function to communicate with our _local_ lookit instance/database. It's possible to run a local instance of Lambda (with AWS and SAM CLIs), but then we'd run into the same problem with the communication between _external_ AWS S3 storage and _local_ Lambda. Therefore the best solution might be to also run a local instance of S3 storage for videos in the dev environment (which would trigger the local Lambda function and Lookit-API call). We could set this up with new docker containers.
As part of this issue, we should also update the documentation for developers/contributing.
**Acceptance Criteria**
- [ ] Scoping
- [ ] Consider Alternatives
- [ ] Solution allows us to locally use and test changes to the Lambda function
- [ ] Solution allows us to locally develop and test all aspects of the experiment runner and lookit-api
- [ ] Solution is as similar as possible to that used in staging/production (to prevent unexpected problems in those environments)
- [ ] May need to ensure that the solution does not cause problems for local storage space (e.g. regular scheduled clean-up, or Make command to clear video files?)
- [ ] Update documentation for developing with RecordRTC rather than Pipe
Contributor guide
Research direction
No files or tests are named in the issue; begin by tracing the RecordRTC-to-S3-to-Lambda-to-Lookit-API flow and reviewing the existing Pipe development setup. Define and document a local solution that supports Lambda, experiment-runner, and lookit-api testing while remaining close to staging/production and addressing video cleanup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, python
- Domain
- backend-api-design, cloud, devops, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100