gradio-app / gradio-app/fastrtc

(likely bug) WebRTC Video track constraints dimensions

Open
#233 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
4.6k
Forks
433
PR merge metrics
No merged PRs in 30d

Description

In order to get a streamed input of NDArray[Shape["H, W, 3"]], the values for width and height needs to be flipped as follows:

```
input_img = WebRTC(label="Stream",
track_constraints={
"width": {"exact": H},
"height": {"exact": W},
},
rtp_params={"degradationPreference": "maintain-resolution"},
mode="send-receive",
modality="video",
mirror_webcam=False)
```

In the below example, I used H=960, W=720 (portrait orientation), following the flipped WebRTC track_constraints configuration as described above. It results in a fully formed portrait image (verified via cv2.imshow on the server), and a stream component/widget of the same portrait aspect ratio. However, the image rendered is consistently chopped off/offset up.

![Image](https://github.com/user-attachments/assets/a3c98aa0-4aaf-4cd1-87a6-bff9cc8941bd)

To summarize the issues:

1. flipped constraints
2. viewport cut-off

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the portrait case with H=960 and W=720, using the shown WebRTC track_constraints and checking both the server-side image and rendered stream widget. Trace how the Stream component applies track constraints and sizes its viewport; done means portrait input renders without requiring flipped width and height values and without cropping or offsetting the image.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
audio-video-rtc, frontend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.