RenderStartup causes an additional frame of latency in rendering (aka pink screen part 2)
- Dominant language
- Rust
- Stars
- 48.2k
- Forks
- 4.8k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 171
Description
This is a followup to #20318. In #20595, we changed how our cameras render slightly to at least change the pink screen into a black screen, which makes the missing frame slightly less noticable.
I documented the cause in other words [here](https://github.com/bevyengine/bevy/issues/20318#issuecomment-3146644481), however I will rewrite it here to hopefully better explain the issue.
---
In order to render with a shader in one frame, these things need to happen in this order:
1) Call `asset_server.load("my_shader");`
1) This starts the task that will load the shader.
2) The load task needs to complete.
1) Shaders are generally cheap to load (especially with embedded assets), so I will assume this always happens.
3) The `handle_internal_asset_events` system needs to run (this is scheduled in `PreUpdate` **in the main world**).
1) This takes the shader asset from the load task and adds it to the ECS.
4) The `ExtractSchedule` must run.
1) This moves the shader asset **from the main world to the render world**
5) The `Render` schedule must run.
1) This first compiles the render pipelines (using the shaders), then runs your rendering system.
2) Note that compiling a render pipeline takes time, but `PipelineCache::block_on_render_pipeline` allows your rendering system to wait for the pipeline to be ready. **Blocking on the render pipeline only works if the shader assets are already in the render world**.
In the current state, the order is:
1) The first main world frame runs. Nothing happens.
1) So far nothing rendering related has run, so no shader loads have even started.
2) `handle_internal_asset_events` runs, but doesn't pick up any shader assets, since we never started any loads.
2) As part of "extract" `RenderStartup` runs.
1) It finally calls `asset_server.load("my_shader");`. For argument's sake, let's assume the load finishes instantly.
3) The `ExtractSchedule` runs.
1) There are no shader assets to move from the main world to the render world, since `handle_internal_asset_events` didn't add any shaders to the ECS.
4) The `Render` schedule runs.
1) We try to compile the render pipelines, but they are missing shader assets, so they can't compile!
2) Rendering systems do nothing when calling `PipelineCache::block_on_render_pipeline` since the shader assets are missing.
3) **We render a black screen!**
5) The main world starts a new frame. From here, everything goes back to normal: the shader asset is added to the ECS, then gets moved to the render world, then the pipelines are compiled, blocked on, and rendered with.
To fix this, we need to either 1) run `RenderStartup` earlier -- e.g., #20407, or 2) have shaders live entirely in the render world (which eschews the entire asset system) -- which could be done as part of WESL changes maybe?
Contributor guide
Research direction
Start by reading the explanation linked in #20318 and the proposed earlier RenderStartup ordering in #20407. Trace handle_internal_asset_events, RenderStartup, ExtractSchedule, and the Render schedule, including PipelineCache::block_on_render_pipeline. Done means the first frame can load, extract, compile, and render the shader without the extra black or pink frame.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- computer-graphics, game-dev
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100