Chocobozzz / Chocobozzz/PeerTube
Picture-in-picture interpreting/translation
- Dominant language
- TypeScript
- Stars
- 15.3k
- Forks
- 1.8k
- Avg merge
- 8d 8h
- Merged PRs (30d)
- 2
Description
I haven't been able to find out much if PeerTube supports picture-in-picture or not. I know it's possible on Blu-Rays, DVDs, within media players on your own desktop and various news websites which are bootstrapped.
The reason why I am making this request is because PeerTube added language recognition for sign languages in [#420
](https://github.com/Chocobozzz/PeerTube/pull/420) in response to [#347](https://github.com/Chocobozzz/PeerTube/issues/347).
The issue is that for many signing deaf folks, subtitles and closed captions are not accessible to them. Not everyone is multilingual, or may have asymmetrical fluency. Some kind of sign language implementation is required.
### Previous attempts of rectifying the issue
There are attempts to create a unique writing system, but usually everyone goes "well, I can read (blank language), why do I need to learn new writing system?" No one has actually sat down to digitalize the writing systems. There was a volunteer named Daniel Danos working on an open-source font but he vanished without explanation and didn't leave his project files behind for others to pick up.
(For more information on history of sign language writing, including its roots in France see here: https://www.handspeak.com/learn/index.php?id=403)
At the moment, the most accessible option for sign language viewers is to add a picture-in-picture stream of a person in the corner of the screen. You see this throughout the world via government news and such.
### Current solutions and their pros/cons
Currently, the best solution is to export the two video files as one as seen here: https://www.youtube.com/watch?v=TCq3ru9HQSc. But this is a short-gap solution and directing certain viewers to a video hosted elsewhere is not inclusive when everyone could be watching the same video with their own needs met within their own clients. You should be able to turn on and off languages you want or don't want, whether it's audible, visual or written.
You can see picture-in-picture in action in this video demonstrating American Sign Language support for Ice Age the movie on Blu-Ray: https://www.youtube.com/watch?v=2lC0uiX_Pz8.
### Documentations
Here is the W3C documentation for pre-recorded synchronized media: https://www.w3.org/WAI/WCAG21/Understanding/sign-language-prerecorded.html. The W3C referenced a European initiative from 1998 which can be found here: https://www.sign-lang.uni-hamburg.de/signingbooks/. The project documentation from Germany is really outdated compared to the Blu-Ray inclusion from 2012. And I am pretty sure the world has moved on from using SMIL and RealMedia as well.
There are probably other use cases where this feature would be beneficial beyond the scope of what I am proposing here.
Contributor guide
Research direction
The issue does not name files, tests, or entry points. Start by determining how PeerTube currently represents alternate audio, video, or subtitle tracks, then define the client and media requirements for selectable picture-in-picture sign-language streams and a testable completion scope.
Written by the indexing model from the issue text.
Assessment
- Domain
- accessibility, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100