Add support for real time audio streaming
- Dominant language
- Rust
- Stars
- 48.2k
- Forks
- 4.8k
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 161
Description
## What problem does this solve or what need does it fill?
During the development of my plugin ([`bevy_fundsp`], also shameless self promotion lol), I found that bevy_audio (and [`bevy_kira_audio`], for that matter) is rather limited.
To play sounds, one must create an `AudioSource`, which stores the bytes of the sound data, then play it in a system using `Res`.
This isn't feasible when using DSP (digital signal processing) libraries such as [`fundsp`](https://github.com/SamiPerttu/fundsp). DSP graphs can be played forever (see example [here](https://github.com/SamiPerttu/fundsp/blob/master/examples/beep.rs)), so it would be impossible to convert these into `AudioSource`s. A workaround for this is to pass a definite length and convert the graph into wave data bytes, which is then converted to an `AudioSource`.
This is very hacky, and it does not exploit the full capabilities of DSP libraries, especially [`fundsp`].
[`bevy_fundsp`]: https://github.com/harudagondi/bevy_fundsp
[`bevy_kira_audio`]: https://github.com/NiklasEi/bevy_kira_audio
[`fundsp`]: https://github.com/SamiPerttu/fundsp
## What solution would you like?
I don't know what exact implementation would actually look like, but I would like:
- A `StreamingAudioSource` that holds an iterator whose items is an array of floats (f32 ideally) where its length is the number of channels (usually two for left/right channels).
- `StreamingAudioSource` can only be played (by continuing to iterate), paused (by stopping the iteration), or possibly reset (this is possible with [`fundsp`] specifically).
Using this solution would probably need access to [`cpal`] directly.
[`cpal`]: https://github.com/rustaudio/cpal
## What alternative(s) have you considered?
Bypass `bevy_audio` and directly use [`cpal`]. This is bad, because [audio programming is very hard](https://tesselode.github.io/articles/audio-libraries-considered-challenging/), and it is better for Bevy to provide its own solution.
Contributor guide
Assessment
This issue has not been assessed yet.