bevyengine / bevyengine/bevy

Add support for real time audio streaming

Open
#5,422 9 comments 7 reactions 0 assignees View on GitHub
A-Audio C-Feature
Dominant language
Rust
Stars
48.2k
Forks
4.8k
Avg merge
3d 22h
Merged PRs (30d)
161

Description

## What problem does this solve or what need does it fill?

During the development of my plugin ([`bevy_fundsp`], also shameless self promotion lol), I found that bevy_audio (and [`bevy_kira_audio`], for that matter) is rather limited.

To play sounds, one must create an `AudioSource`, which stores the bytes of the sound data, then play it in a system using `Res`.

This isn't feasible when using DSP (digital signal processing) libraries such as [`fundsp`](https://github.com/SamiPerttu/fundsp). DSP graphs can be played forever (see example [here](https://github.com/SamiPerttu/fundsp/blob/master/examples/beep.rs)), so it would be impossible to convert these into `AudioSource`s. A workaround for this is to pass a definite length and convert the graph into wave data bytes, which is then converted to an `AudioSource`.

This is very hacky, and it does not exploit the full capabilities of DSP libraries, especially [`fundsp`].

[`bevy_fundsp`]: https://github.com/harudagondi/bevy_fundsp
[`bevy_kira_audio`]: https://github.com/NiklasEi/bevy_kira_audio
[`fundsp`]: https://github.com/SamiPerttu/fundsp

## What solution would you like?

I don't know what exact implementation would actually look like, but I would like:

- A `StreamingAudioSource` that holds an iterator whose items is an array of floats (f32 ideally) where its length is the number of channels (usually two for left/right channels).
- `StreamingAudioSource` can only be played (by continuing to iterate), paused (by stopping the iteration), or possibly reset (this is possible with [`fundsp`] specifically).

Using this solution would probably need access to [`cpal`] directly.

[`cpal`]: https://github.com/rustaudio/cpal

## What alternative(s) have you considered?

Bypass `bevy_audio` and directly use [`cpal`]. This is bad, because [audio programming is very hard](https://tesselode.github.io/articles/audio-libraries-considered-challenging/), and it is better for Bevy to provide its own solution.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.