WASM audio too much `play` calls for AudioContext
- Dominant language
- Rust
- Stars
- 48.2k
- Forks
- 4.8k
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 161
Description
## Bevy version
`0.11.0`
## What you did
I compile my game to WASM.
When I try to play my games on mobile browser I noticed intemittent/broken sound.
On desktop browser everything "works" well.
## Additional information
I investigated the issue.
First of all I created some patches `audio_context_patch.js` for original [AudioContext](https://developer.mozilla.org/ru/docs/Web/API/AudioContext)
```javascript
const audioContextList = [];
function constructorPatch(obj, key) {
obj[key] = new Proxy(obj[key], {
construct(target, args) {
const audioContext = new target(...args);
audioContextList.push(audioContext);
return audioContext;
},
apply(target, thisArg, args) {
console.log(`patch apply ${key}`);
return Reflect.apply(target, thisArg, args);
},
});
}
constructorPatch(globalThis, 'AudioContext');
constructorPatch(AudioBufferSourceNode.prototype, 'start');
const userInputEventNames = [
"mousedown",
"pointerdown",
"touchdown",
"keydown",
];
function resumeAudioContexts() {
console.log('resumeAudioContexts');
let count = 0;
audioContextList.forEach((context) => {
if (context.state !== "running") {
context.resume();
} else {
count++;
}
});
if (count > 0 && count === audioContextList.length) {
userInputEventNames.forEach((eventName) => {
document.removeEventListener(eventName, resumeAudioContexts);
});
}
}
userInputEventNames.forEach((eventName) => {
document.addEventListener(eventName, resumeAudioContexts);
});
```
Above code does 2 things:
1) Resume `AudioContext` when user "interacted" with the page some how
2) Make interceptor/patch for `play()` method for `AudioBufferSourceNode`
For example you can compile to WASM [the simplest audio example](https://github.com/bevyengine/bevy/blob/v0.11.0/examples/audio/audio.rs):
```rust
//! This example illustrates how to load and play an audio file.
use bevy::prelude::*;
fn main() {
App::new()
.add_plugins(DefaultPlugins)
.add_systems(Startup, setup)
.run();
}
fn setup(asset_server: Res, mut commands: Commands) {
commands.spawn(AudioBundle {
source: asset_server.load("sounds/Windless Slopes.ogg"),
..default()
});
}
```
compile to WASM is pretty straightforward:
- Add WebAssembly support to your Rust installation
```sh
rustup target install wasm32-unknown-unknown
```
- Install [wasm-bindgen](https://github.com/rustwasm/wasm-bindgen) CLI
```sh
cargo install wasm-bindgen-cli
```
- compile to WASM
```sh
wasm-bindgen --no-typescript --out-name wasm_bindgen_core --out-dir dist/ --target web target/wasm32-unknown-unknown/release/....your_file_name....wasm
```
- run compiled example with following `index.html`:
```html
Example
```
notice that `audio_context_patch.js` must be before regular `wasm_bindgen_core.js` to apply patches.
As a result in browser console I see following:

`AudioBufferSourceNode.play()` keeps calling each frame again and again!
This is too overwhelming for mobile browser, that is why I hear corrupted sound.
This is appropriate for desktop browser some-how, probably because of performant CPU.
Navigating through the call stack of WASM shows not much for me:
```
real (wasm_bindgen_core.js:199)
requestAnimationFrame (async)
(anonymous) (wasm_bindgen_core.js:399)
handleError (wasm_bindgen_core.js:227)
imports.wbg.__wbg_requestAnimationFrame_d082200514b6674d (wasm_bindgen_core.js:398)
$winit::platform_impl::platform::event_loop::runner::Shared::apply_control_flow::hf7f85fd12e8e3b67 (wasm_bindgen_core_bg.wasm:0x4218f2)
$winit::platform_impl::platform::event_loop::runner::Shared::run_until_cleared::h52fc249ce9680fb7 (wasm_bindgen_core_bg.wasm:0x4e3100)
$winit::platform_impl::platform::backend::timeout::AnimationFrameRequest::new::{{closure}}::hf77d92138eacb4da (wasm_bindgen_core_bg.wasm:0x783754)
$wasm_bindgen::convert::closures::invoke0_mut::hf7a6e5275be6b226 (wasm_bindgen_core_bg.wasm:0x7b5f70)
...
and the same loop again and again
...
```
I didn't investigate bevy `0.10.1` the same way however I heared the same corrupted sound in mobile browser - so I think this issue persisted a long time ago, nobody tested sound in mobile browser.
Contributor guide
Assessment
This issue has not been assessed yet.