Automattic / Automattic/VU-VRM

Updates Please! - Integrate with AI

Open
#2 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
106
Forks
22
PR merge metrics
No merged PRs in 30d

Description

Hi there
i gotta tell you, its not easy finding a working out of the box
realtime audio 2 face that just works

i see that its been a while since this repo got some love
was wondering if you'd like to revisit this
In the current age of ai this is more useful than ever

would love for this to also work with plain 2d images
and perhaps make it better - with real expressions, eye blinks, etc
any improvement will be awesome - which should be easier than ever to do today - get help from ai

and especially, on how to use it with llms... i can convert the text response to audio pretty quickly,
and would like to pass it to VU-VRM instead of the mic please

thanks and all the best

PS: here are other findings,
https://github.com/YuanxunLu/LiveSpeechPortraits
https://github.com/Kedreamix/Awesome-Talking-Head-Synthesis - good for research
https://github.com/antgroup/ditto-talkinghead - best available realtime i found so far

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are identified. Begin by surveying the repository to locate the microphone-to-VRM audio path, then clarify whether 2D images, facial expressions, LLM-generated audio, and the listed research integrations are in scope; done is an agreed, testable scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ai, audio-video-rtc, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.