Tencent-RTC / Tencent-RTC/TRTC_Web
[Feature] FaceDetection plugin should support real inference FPS and input resolution control
@Rychou is already working on this.
Since Jul 3, 2026.
- Dominant language
- JavaScript
- Stars
- 160
- Forks
- 72
- Avg merge
- 6m
- Merged PRs (30d)
- 2
Description
Is your feature request related to a problem? Please describe.
The current FaceDetection plugin appears to run face detector inference on every video frame, while detectionInterval only throttles state handling / callbacks. This makes it difficult to use FaceDetection in low-end devices or TRTC applications that need very low overhead face-presence detection.
From the published trtc-sdk-v5@5.18.2 package:
plugins/video-effect/face-detection/face-detection.esm.js:detectFace()callsgetCurrentStatus()before checking_detectionInterval.getCurrentStatus()callsthis._visionTaskRegistry.getResult(this._faceDetectorHash).detectionsand thenresetHashResults().assets/mediapipe/vision.js:FaceDetectoris mapped toinstance.detectForVideo.bind(instance).VisionTaskRegistry.reasoning()calls the reasoning function withvideoandperformance.now().detectFace()schedules itself again withvideo.requestVideoFrameCallback(this.detectFace.bind(this)).
So with a 30 FPS camera stream, detectForVideo may run close to 30 times per second even if detectionInterval is set to a much larger value.
Describe the solution you'd like
Please add options to the FaceDetection plugin to control actual inference workload:
inferenceFpsorfps: limit how oftendetectForVideois called, for example 1-5 FPS for lightweight face-presence checks.inputResolution: allow downsampling before inference, for example{ width: 96, height: 72 },{ width: 160, height: 120 }, or similar.- Keep
detectionIntervalfor callback/state debounce, or clarify its documentation if it is not intended to throttle model inference.
A possible implementation would be:
- Move the time check before
getCurrentStatus(), so expensive inference is skipped until the next inference window. - Optionally use a canvas / OffscreenCanvas as the FaceDetector input when
inputResolutionis provided. - Preserve the current behavior as the default to avoid breaking existing users.
Describe alternatives you've considered
Applications can implement a custom MediaPipe FaceDetector pipeline with low-resolution sampling and low FPS, but this duplicates functionality already provided by the TRTC plugin and requires separate handling of camera/video resources.
Additional context
This is useful for applications that already use TRTC and only need a simple hasFace: boolean signal, especially when running alongside audio/video publishing on low-end clients. In such cases, 1-2 FPS and a small input resolution are often enough and can significantly reduce CPU usage.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.