w3c / w3c/webcodecs

File System Access API + WebWorker + Recording Webm video and audio sample

Open
#446 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

samples
Dominant language
HTML
Stars
1.3k
Forks
194
Avg merge
1d 13h
Merged PRs (30d)
3

Description

Is there a sample that can encode a ReadableStream transferred to a worker into a webm format with audio and stream it to the file system using the new File System Acccess API? I was able to use one of the samples which has a modified webm-writer2.js file and achieve everything video related on a worker, but I am stuck on finding an encoder and muxer that can do both audio and video streams in the worker using the new WebCodecs API.

This example has a workaround for audio, but it relies on AudioContext which can only be used on the main thread. This can become problematic and cause interruptions on the main loop as it needs to constantly read chunks from an event.
https://github.com/xiph/opus/issues/221

My goal is to record a video/audio stream, transfer the MediaStream entirely off the main thread for processing and stream, mux it all into a single webm container, and save it to the file system without any memory limits. Does anyone have a working solution that can build off that sample and include audio?

Here's some references:

webm-writer2.js (video only so far)
https://github.com/w3c/webcodecs/tree/main/samples/capture-to-file

Audio encoding problem?
https://github.com/xiph/opus/issues/221

Transferring media stream to worker via track processor
https://github.com/w3c/webcodecs/issues/43

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the capture-to-file sample and its modified webm-writer2.js, then review the linked WebCodecs issue about transferring media streams and the Opus audio-encoding issue. A complete solution would process transferred audio and video in a worker, mux them into WebM, and stream the result to the File System Access API without memory limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
audio-video-rtc, web-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.