Bevy needs better support for pre-multiplied alpha textures and sprites
- Dominant language
- Rust
- Stars
- 48.2k
- Forks
- 4.8k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 171
Description
## What problem does this solve or what need does it fill?
To blend translucent textures correctly, it is nearly always necessary to store them with the alpha channel *pre-multiplied* so that linear interpolation will produce correct values. `BlendState::PREMULTIPLIED_ALPHA_BLENDING` is required to blend these pre-multiplied textures correctly (exposed by bevy in the PBR pipeline as `AlphaMode::Premultiplied`).
Bevy currently has very poor support for doing this consistently. Some render pipelines hard-code a non-premultiplied blend mode, there's no easy way to convert textures from straight alpha to pre-multiplied alpha, and what a texture is expected to use when this isn't configurable is nearly completely undocumented.
Given that many sprites in 2D games have at least some translucent pixels around the edges (unless you're strictly doing pixel art) even when their content is largely opaque, this seems important to get right.
## What solution would you like?
Bevy should consistently use pre-multiplied alpha for rendering transparent objects *by default* and provide an easy way to convert textures loaded from disk to pre-multiplied alpha, ideally doing this automatically during load for disk formats that are always using straight alpha (like PNG) or that encode whether they are pre-multiplied.
There should be no render pipelines that render user-controlled textures that hard-code the blend mode to use straight alpha. Currently there are at least these:
The mesh2d pipeline:
https://github.com/bevyengine/bevy/blob/7f658cabf7abd98d52bd94db8bd7fc788f49856a/crates/bevy_sprite/src/mesh2d/mesh.rs#L548
The sprite pipeline:
https://github.com/bevyengine/bevy/blob/7f658cabf7abd98d52bd94db8bd7fc788f49856a/crates/bevy_sprite/src/render/mod.rs#L283
The default UI pipeline:
https://github.com/bevyengine/bevy/blob/7f658cabf7abd98d52bd94db8bd7fc788f49856a/crates/bevy_ui/src/render/pipeline.rs#L95
The upscale node also does this, which doesn't seem like it will produce the correct result if the render target uses pre-multiplied alpha:
https://github.com/bevyengine/bevy/blob/7f658cabf7abd98d52bd94db8bd7fc788f49856a/crates/bevy_core_pipeline/src/upscaling/mod.rs#L50
Gizmos also do this. It matters less there since they're mostly for debugging and don't have textures.
## What alternative(s) have you considered?
- Keep forcing some pipelines to use straight alpha, potentially creating artifacts.
- Don't provide a way to convert loaded sprites to pre-multiplied alpha, making it the user's problem.
## Additional context
This 9 years old article by Tom Forsyth also argues in favor of pre-multiplied alpha in engines: https://tomforsyth1000.github.io/blog.wiki.html#%5B%5BPremultiplied%20alpha%5D%5D
It optimistically suggests:
> So anyway, yeah, premultiplied alpha. Use it, love it, pass it on. Then maybe we'll only take another 20 years til everyone's doing this stuff correctly.
Given the age of the article bevy still has over 10 years to get this right.
Also, things could still be worse. For example, OBS, to this day, **defaults** to what it euphemistically calls "*non-linear alpha blending*" for image sources, for compatibility with assets created for old OBS versions that had a bug where the alpha channel was subjected to the sRGB transfer function by accident.
Contributor guide
Research direction
Start by reviewing the hard-coded blend modes in crates/bevy_sprite/src/mesh2d/mesh.rs, crates/bevy_sprite/src/render/mod.rs, crates/bevy_ui/src/render/pipeline.rs, and crates/bevy_core_pipeline/src/upscaling/mod.rs. Then trace how disk textures are loaded and how alpha conventions are documented. Done means the affected pipelines and texture-loading behavior have a consistent, clearly documented premultiplied-alpha policy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- computer-graphics, game-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100