FIRST-Tech-Challenge / FIRST-Tech-Challenge/scorekeeper

Make FTC Live capable of doing video compositing for livestreams

Open
#719 0 comments 2 reactions 0 assignees View on GitHub
enhancement Local
Dominant language
No language data
Stars
110
Forks
26
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
I realize this is a longshot request, but I think it would be very beneficial for smaller events that don't have as many volunteers and resources, and can lead to a more consistent experience for all those involved. I would like to see the ability added to do video compositing and mixing to create a single browser view that can be streamed directly to YouTube without requiring multiple scenes or any fancy setup in the streaming software.

**Describe the solution you'd like**
This basically builds on top of my #718 request. I have split this request into two parts. First is the minimum implementation which I feel is the minimum number of features that would need to be added to make this useful for at least smaller events. And then I have my full implementation request which is the features I would need to see to use this at larger events and as a replacement for the current way we setup events.

*Minimum implementation:*
1) Have the ability to plug USB cameras into client computers connected to the system (e.g. the timer computers, or dedicated camera computers).
2) Have the ability for the client computers to send the camera stream back to the server via a WebRTC stream (this *might* require #717)
3) Have the ability to assign each camera to one or more roles (Field 1, Field 2, Ceremonies, Alliance Selection, etc.)
4) Add a video stream display option that is the same as the current Overlay display, but instead of having the chroma key layer it has a Video Viewer
5) Automatically switch the video stream to the correct camera based on the match state

With this it would allow most events that have internet connection to stream their event with the only additional cost being a couple of webcams and tripods. It doesn't require any special technical knowledge or experience setting up livestream software beyond a simple browser capture and a streaming target (YouTube or Twitch). It eliminates the need of setting up layers in multiple scenes and then either manually switching between scenes or using third-part tools to automate switching of scenes.

*Full implementation:*
In addition to the requested features above, also include
1) Ability to pull in [NDI](https://ndi.tv/) camera streams
2) Ability to add picture-in-picture for sign language interpreter
3) If next years game has any element outside the field like the landing zones this year, ability to add a second camera feed for that
4) Standardized recommendations for camera placements to create consistency across regions and events
5) Easy way to pull in announcer audio directly into the system rather than doing that in the streaming software (this might not even be possible)
6) Generate a NDI output stream of the mixed composited video (Very low priority)

**Describe alternatives you've considered**
Currently everything I have requested is possible and done using a mix of other tools. The compositing can be done using streaming software such as OBS Studio or XSplit. Videos can be played in the streaming software. I have written a script that connects to the WebSocket API in FTC Live and OBS to automate scene switching when match preview is shown. However, all of this requires a certain level of experience and knowledge that not every region or event has. What I am proposing can be very easy to setup, and then in OBS all you would need is a simple browser capture, maybe audio capture device, and you would be ready to stream.

**Additional context**
N/A

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.