huggingface / huggingface/transformers.js
[Question] Any examples of processing video frames of a user uploaded video (specifically for depth estimation)?
- Dominant language
- JavaScript
- Stars
- 16.3k
- Forks
- 1.2k
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 6
Description
Hi there, I'm wondering if there are any examples of processing video frames of a user uploaded video? I'm specifically looking to run depth estimation on each frame of a short video, but any similar example would be useful.
If not, does this approach seem correct?
* Use one of the approaches described [here](https://stackoverflow.com/questions/32699721/javascript-extract-video-frames-reliably) to draw each frame of the video to a canvas
* Call `HTMLCanvasElement.toBlob()` on the canvas to get a `Blob`
* Pass N (10?) of those Blobs to a worker at a time
* For each of those Blobs call `const image = await RawImage.fromBlob(blob)` to get a `RawImage`
* Run depth estimation on the list of images with `await classifier([rawImage1, rawImage2, etc.])`
Thanks for any help!
Contributor guide
Assessment
This issue has not been assessed yet.