Support asynchronous Picking backends
- Dominant language
- Rust
- Stars
- 48.2k
- Forks
- 4.8k
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 161
Description
## What problem does this solve or what need does it fill?
We have the following constraints:
- We display very large 3D models stored in a remote server.
- We can't store the 3D mesh in RAM so we set `RenderAssetUsages::RENDER_WORLD` on our Meshes.
We want to detect mouse/touch clicks on Nodes to "select" them (show an outline and perform other operations on them later on).
Today, we can't use `MeshPickingPlugin` because it relies on the CPU-side raycast which doesn't work because we don't store the Meshes in the `MAIN_WORLD`.
We tried making a custom backend, but it's supposed to be called every frame, even when there's no mouse click, which we can't do because it would lead to hundreds of API calls.
## What solution would you like?
Support asynchronous picking backends that can be executed only when needed like on mouse clicks, not on every frame.
## What alternative(s) have you considered?
We were considering the following:
- Make our custom backend trigger a raycast only when there's a mouse click, then repeat the `PointerHits` events a few frames so [`pointer_events`](https://github.com/bevyengine/bevy/blob/73970d0c1215ee2f290513da7aed2e619a77bd7b/crates/bevy_picking/src/events.rs#L418) thinks there has been a click.
- This doesn't work because the mouse click has already happened by the time [`pointer_events`](https://github.com/bevyengine/bevy/blob/73970d0c1215ee2f290513da7aed2e619a77bd7b/crates/bevy_picking/src/events.rs#L418) receives the `PointerHits`.
- Not using the picking of Bevy at all.
- Switch to a "color picking shader" method instead, as described in https://github.com/aevyrie/bevy_mod_picking/issues/3.
## Additional context
We tried doing something like this to do the raycast only after a mouse click:
```rs
#[allow(clippy::too_many_arguments)]
pub fn update_hits(
ray_map: Res,
picking_cameras: Query<&Camera>,
mut ray_cast: MeshRayCast,
mut output: EventWriter,
mouse_button_input: Res>,
) {
if !mouse_button_input.just_released(MouseButton::Left) {
return;
}
// Do the async raycast here.
// Send the event later.
commands.send_event(PointerHits::new(...));
}
// Repeat the PointerHits event to simulate the event being triggered for 2 frames.
fn repeat_pointer_hits(
mut pointer_hits: EventReader,
mut commands: Commands,
mut event_sent: Local,
) {
if *event_sent {
*event_sent = false;
return;
}
for ev in pointer_hits.read() {
println!("repeat_pointer_hits: Pointer hits: {:?}", ev.clone());
commands.send_event(ev.clone());
*event_sent = true;
}
}
```
Contributor guide
Research direction
Start by reading the MeshPickingPlugin flow and the pointer_events logic in crates/bevy_picking/src/events.rs, then trace how picking backends produce PointerHits. Done should allow a backend to perform work only when requested, including after a delayed asynchronous result, while still producing usable pointer events for selection.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- game-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100