godotengine / godotengine/godot
XRHandModifier3D moves invalid joints to the world origin
- Dominant language
- C++
- Stars
- 117k
- Forks
- 26.8k
- PR merge metrics
- PR metrics pending
Description
### Tested versions
- Reproducible in Godot 4.5.1.stable
### System information
Windows 10 - Godot v4.5.1.stable - Vulkan (Mobile)
### Issue description
When attempting to use OpenXR hand tracking with my Vive Pro Eye, my metacarpal bones are snapped to the world origin!
The Vive Pro Eye [does not](https://developer.vive.com/resources/openxr/openxr-pcvr/tutorials/unreal-engine/integrate-hand-tracking-data-your-hand-model/) track metacarpal bone orientations ~~(but it does track positions)~~.
~~I think this problem appears to primarily be due to the use of an [uninitialized `Transform3D`](https://github.com/godotengine/godot/blob/4.5/scene/3d/xr/xr_hand_modifier_3d.cpp#L229) when a hand joint is determined to be "invalid" - which I assume defaults to all zeroes (hence the world origin). The culprit appears to be [an unhandled `else`](https://github.com/godotengine/godot/blob/4.5/scene/3d/xr/xr_hand_modifier_3d.cpp#L240).~~
I also want to note that the usage of "valid" orientation here seems incorrect to my eye. [Per the OpenXR spec](https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#_locate_hand_joints):
> If the returned `isActive` is true, the runtime **must** return all joint locations with both `XR_SPACE_LOCATION_POSITION_VALID_BIT` and `XR_SPACE_LOCATION_ORIENTATION_VALID_BIT` set. Although, in this case, some joint space locations **may** be untracked (i.e. `XR_SPACE_LOCATION_POSITION_TRACKED_BIT` or `XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT` is unset).
I think this means it is also being sent incorrectly by Vive's extension (which reports unset metacarpal orientation validity bits); but it likely shouldn't be used by Godot as a standin for whether a joint is being tracked or not either, so there are two sides to this undefined-behaviour coin.
But, regardless, if a joint is not deemed to have valid tracking data, it should still remain attached to the hand, and not moved to the world origin 🙂
### Steps to reproduce
- Use an OpenXR runtime with XR_EXT_hand_tracking, which provides some invalid orientation bits (ie. [Vive Console for SteamVR](https://store.steampowered.com/app/1635730/VIVE_Console_for_SteamVR/) on the Vive Pro Eye - I use the beta)
- Verify in SteamVR settings > OpenXR the hand tracking API Layer is installed *(ie. ViveVR OpenXR Hand Tracking)*
- Launch the bare-minimum godot project below
- Confirm that the orientation bit is unset for some bones in the output for your left hand (bit 0 of `Flags`)
- Observe your metacarpal bones distorting to the world origin
### Minimal reproduction project (MRP)
[joint-validity-mrp.zip](https://github.com/user-attachments/files/24033265/joint-validity-mrp.zip)
Contributor guide
Research direction
Start in scene/3d/xr/xr_hand_modifier_3d.cpp around lines 229 and 240, then reproduce the issue with the linked joint-validity-mrp.zip project and an OpenXR runtime that reports invalid orientation bits. Confirm how invalid joints are handled and verify that the metacarpal bones no longer move to the world origin while testing the affected hand-tracking output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ar-vr-xr, game-dev
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100