aws-samples / aws-samples/amazon-sumerian-hosts
Speech animation, gaze tracking, face shape break with BabylonJS v5
- Dominant language
- JavaScript
- Stars
- 210
- Forks
- 83
- PR merge metrics
- No merged PRs in 30d
Description
### Description:
If you attempt to use the Hosts with BabylonJS v5, audio and body animations work properly, but there is no speech animation, gaze tracking stops working, and the shape of the characters head (jaw, chin, and eyes especially) look odd (see attached images.)
### Steps to Reproduce:
To reproduce:
1. Clone the respository.
2. Ensure you have branch “mainline2.0” checked out.
3. Delete the `package-lock.json` file.
4. Open the file `/packages/amazon-sumerian-hosts-babylon/package.json` for editing.
5. Change the “peerDependencies” to the following:
```
"peerDependencies": {
"@babylonjs/core": "^5.0.0",
"@babylonjs/loaders": "^5.0.0"
}
```
6. From the *root of the repository* (important), run `npm install`
7. Edit the `/demo-credentials.js` file to use a Cognito Identity Pool you’ve set up for use with Sumerian Hosts (see `/AWS-Infrastructure-Setup.md` for instructions if you don’t already have this set up.)
8. From the root of the repository, run `npm run start-babylon`
9. Once the browser tab titled “BabylonJS Sumerian Host Demo” loads, select the “Hello World Demo”.
10. Click the “Start” button on the demo.
11. Observe that the that audio plays but the host’s mouth doesn’t move and the host’s jawline/chin/eyes don't look right.
**Expected:**

**Actual:**

Contributor guide
Assessment
This issue has not been assessed yet.