bevyengine / bevyengine/bevy

viewport_picking uses logical_viewport_size for render-to-image cameras, causing DPI-scaled coordinate mismatch

Open
#23,055 0 comments 0 reactions 0 assignees View on GitHub
A-UI C-Bug D-Straightforward S-Ready-For-Implementation
Dominant language
Rust
Stars
48.2k
Forks
4.8k
Avg merge
3d 22h
Merged PRs (30d)
161

Description

Caveat: Report by my friend Claude. He did a lot of thinking, that usually means no "apparent" solution or documentation or bug report existed on how to properly combine DPI, UI scaling, render to texture, bevy picking and viewport node :)

## Bevy version

0.18.0

## What I did

I'm using `ViewportNode` with a camera rendering to an `Image` render target, combined with mesh picking (`MeshPickingCamera` + `Pickable` on meshes), on a display with DPI scale factor > 1.0 (Windows 11, 150% scaling, `scale_factor = 1.5`).

## What went wrong

Mesh picking coordinates are misaligned with rendered mesh positions. Hovering over a visually rendered mesh does not trigger picking — I have to hover at a position offset toward the top-left corner. The offset scales proportionally with the window's DPI scale factor. At `scale_factor = 1.0` the issue does not occur.

## Root cause

In [`viewport_picking`](https://github.com/bevyengine/bevy/blob/v0.18.0/crates/bevy_ui/src/widget/viewport.rs#L61-L152), pointer coordinates are normalized into `[0, 1]` range using the node's logical size, then scaled back up using [`camera.logical_viewport_size()`](https://github.com/bevyengine/bevy/blob/v0.18.0/crates/bevy_ui/src/widget/viewport.rs#L117):

```rust
let local_position = (input.location.position - top_left) / logical_size;
let position = local_position * cam_viewport_size;
```

For a camera rendering to an `Image` via `RenderTarget::Image`, `logical_viewport_size()` returns `physical_image_size / window_scale_factor`. On my 1.5x DPI display with a 1004×833 physical image, this returns 669×555.

However, the downstream mesh picking system ([`ray_cast_to_world`](https://github.com/bevyengine/bevy/blob/v0.18.0/crates/bevy_picking/src/mesh_picking/ray_cast/intersections.rs)) converts this position to NDC using the camera's **physical** viewport size. So a position at 44% across the logical viewport (correct) gets interpreted as 29% across the physical viewport (incorrect).

### The coordinate chain

| Step | Value | Correct? |
|------|-------|----------|
| Pointer in window (logical) | (370, 378) | ✓ |
| Node top-left (logical) | (76, 130) | ✓ |
| Normalized `[0,1]` | (0.44, 0.45) | ✓ |
| `× cam_viewport_size` (669×555) | (294, 248) | **✗** — should be (441, 373) |
| Mesh picking interprets as physical (1004×833) | NDC = (-0.41, 0.40) | **✗** — should be (-0.12, 0.10) |

The normalization step is correct. The bug is that `cam_viewport_size` divides by the window's scale factor, even though the image render target has no concept of DPI scaling.

## Suggested fix

[Line 117](https://github.com/bevyengine/bevy/blob/v0.18.0/crates/bevy_ui/src/widget/viewport.rs#L117) should use the physical target size instead of the logical viewport size:

```diff
- let Some(cam_viewport_size) = camera.logical_viewport_size() else {
+ let Some(cam_viewport_size) = camera.physical_target_size().map(|s| s.as_vec2()) else {
```

DPI scaling is a display concept — it doesn't apply to render-to-texture targets. The image has a fixed pixel size (set by [`update_viewport_render_target_size`](https://github.com/bevyengine/bevy/blob/v0.18.0/crates/bevy_ui/src/widget/viewport.rs#L155-L173) using the node's **physical** size), and picking should address those pixels directly.

## Workaround

Bypass Bevy's mesh picking for the viewport camera entirely and implement custom picking using the window cursor position, scaling by `window.scale_factor()`, and doing manual ray intersection against known mesh positions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.