godotengine / godotengine/godot

XRHandModifier3D moves invalid joints to the world origin

Open
#113,752 8 comments 0 reactions 0 assignees View on GitHub
bug topic:xr
Dominant language
C++
Stars
117k
Forks
26.8k
PR merge metrics
PR metrics pending

Description

### Tested versions

- Reproducible in Godot 4.5.1.stable

### System information

Windows 10 - Godot v4.5.1.stable - Vulkan (Mobile)

### Issue description

When attempting to use OpenXR hand tracking with my Vive Pro Eye, my metacarpal bones are snapped to the world origin!

Image

The Vive Pro Eye [does not](https://developer.vive.com/resources/openxr/openxr-pcvr/tutorials/unreal-engine/integrate-hand-tracking-data-your-hand-model/) track metacarpal bone orientations ~~(but it does track positions)~~.

~~I think this problem appears to primarily be due to the use of an [uninitialized `Transform3D`](https://github.com/godotengine/godot/blob/4.5/scene/3d/xr/xr_hand_modifier_3d.cpp#L229) when a hand joint is determined to be "invalid" - which I assume defaults to all zeroes (hence the world origin). The culprit appears to be [an unhandled `else`](https://github.com/godotengine/godot/blob/4.5/scene/3d/xr/xr_hand_modifier_3d.cpp#L240).~~

I also want to note that the usage of "valid" orientation here seems incorrect to my eye. [Per the OpenXR spec](https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#_locate_hand_joints):

> If the returned `isActive` is true, the runtime **must** return all joint locations with both `XR_SPACE_LOCATION_POSITION_VALID_BIT` and `XR_SPACE_LOCATION_ORIENTATION_VALID_BIT` set. Although, in this case, some joint space locations **may** be untracked (i.e. `XR_SPACE_LOCATION_POSITION_TRACKED_BIT` or `XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT` is unset).

I think this means it is also being sent incorrectly by Vive's extension (which reports unset metacarpal orientation validity bits); but it likely shouldn't be used by Godot as a standin for whether a joint is being tracked or not either, so there are two sides to this undefined-behaviour coin.

But, regardless, if a joint is not deemed to have valid tracking data, it should still remain attached to the hand, and not moved to the world origin 🙂

### Steps to reproduce

- Use an OpenXR runtime with XR_EXT_hand_tracking, which provides some invalid orientation bits (ie. [Vive Console for SteamVR](https://store.steampowered.com/app/1635730/VIVE_Console_for_SteamVR/) on the Vive Pro Eye - I use the beta)
- Verify in SteamVR settings > OpenXR the hand tracking API Layer is installed *(ie. ViveVR OpenXR Hand Tracking)*
- Launch the bare-minimum godot project below
- Confirm that the orientation bit is unset for some bones in the output for your left hand (bit 0 of `Flags`)
- Observe your metacarpal bones distorting to the world origin

### Minimal reproduction project (MRP)

[joint-validity-mrp.zip](https://github.com/user-attachments/files/24033265/joint-validity-mrp.zip)

Contributor guide

Open the contributing guide

Research direction

Start in scene/3d/xr/xr_hand_modifier_3d.cpp around lines 229 and 240, then reproduce the issue with the linked joint-validity-mrp.zip project and an OpenXR runtime that reports invalid orientation bits. Confirm how invalid joints are handled and verify that the metacarpal bones no longer move to the world origin while testing the affected hand-tracking output.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ar-vr-xr, game-dev
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.