runwayml / runwayml/avatars-sdk-react

Custom avatar with CUSTOM voice renders the wrong face (stock-like fallback) in gwm1_avatars realtime sessions; preset-voice clone of same reference renders correctly

Open
#38 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
22
Forks
11
PR merge metrics
No merged PRs in 30d

Description

Summary

A custom Character configured with a custom voice consistently renders a different face than its stored reference/processed image in gwm1_avatars realtime sessions. An otherwise-identical Character (same reference image) using a preset voice renders the correct face. This looks like the custom-voice configuration routing through a path that fails to load the custom avatar and falls back to a stock-like face.

Reproduction (ours, repeated across ~10 sessions in one day)

  1. Character ddc7d594-b940-42e4-8754-d1e4d37fca13 (name COLTRANE, custom voice id 10fd1e64-dbc7-43da-8414-b77b70889342, status READY). Its referenceImageUri and processedImageUri (verified via GET /v1/avatars/:id) show the intended face.
  2. Create a realtime session: POST /v1/realtime_sessions with {model: 'gwm1_avatars', avatar: {type:'custom', avatarId:'ddc7d594-…'}, livekit:{…}} (puppet mode — we publish audio, subscribe the avatar A/V).
  3. The published avatar video is a different person than the stored reference — every session. Example session ids from 2026-07-02 (US MST): 86336814-09eb-47c6-8309-0ad404e28fb8, dc79d811-… era runs, da69b3b0-022b-4184-8809-2435ada5ac1d, dabad714-eadb-4d00-a996-d5ce1fe829b4.

Discriminating experiment

Created a5bf9339-2f48-4c60-aca9-78205a39a89a (COLTRANE-PRESET-AB) from the same reference image, with preset voice jasper — session dc79d811-d0a0-42d6-a066-8937dc9e92de rendered the correct face immediately. Same night, same integration, same room. Only variable: voice type.

Possibly related: docs note webcam/screen-share is unsupported for custom Characters with custom voices — the same configuration that fails here.

Environment

  • API version header 2024-11-06, raw REST integration (also reproduced via @livekit/agents-plugin-runway path)
  • Happy to provide org id, more session ids, or run a live repro.

Expected

Custom avatar + custom voice renders the stored processed image, or the session errors loudly instead of silently rendering a different identity — for avatars representing real/branded personas, silently wearing another face is a serious failure mode.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing POST /v1/realtime_sessions with the reported custom avatar and custom voice, then compare it with the same reference image using preset voice jasper. Check the @livekit/agents-plugin-runway path and the custom-character webcam/screen-share limitation. Done means the custom voice session renders the stored processed image or fails explicitly instead of showing another face.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, audio-video-rtc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.