KitwareMedical / KitwareMedical/SlicerVirtualReality

Refactor transform tracking to support OpenXR

Open
#122 0 comments 0 reactions 0 assignees View on GitHub
Backend: OpenXR enhancement
Dominant language
C++
Stars
137
Forks
62
Avg merge
8h 8m
Merged PRs (30d)
8

Description

## OpenVR

Transforms for controllers, HMD, and trackers can optionally be created and added to the scene.

The update of the transform is done using the following API:

https://github.com/KitwareMedical/SlicerVirtualReality/blob/53a9549ce19b398d9f0dcca158deaa653ddf91e1/VirtualReality/Widgets/qMRMLVirtualRealityView_p.h#L92-L95

all transform nodes have the following attributes in common:

```
VirtualReality.PoseValid (`True` or `False`)
VirtualReality.PoseStatus (`Uninitialized`, `CalibratingInProgress`, `CalibratingOutOfRange`, `RunningOk`, `RunningOutOfRange`)
```

### Controllers

Based on the state of the controller, the following MRML attributes are set:

```
VirtualReality.ControllerActive
VirtualReality.ControllerConnected
```

Additionally, the position of each controller is updated leveraging the OpenVR specific API `vtkOpenVRRenderWindow::GetOpenVRPose()`

### HMD

Based on the retrieve state, the attribute `VirtualReality.HMDActive` is set:

https://github.com/KitwareMedical/SlicerVirtualReality/blob/53a9549ce19b398d9f0dcca158deaa653ddf91e1/VirtualReality/Widgets/qMRMLVirtualRealityView.cxx#L576-L583

### Generic trackers

In additional of the common attributes, the transform nodes for the generic trackers are named "VirtualReality.GenericTracker" and set with these attributes:

```
VirtualReality.VRDeviceID
VirtualReality.TrackerActive
```

Related issues:
* https://github.com/KitwareMedical/SlicerVirtualReality/issues/171

## OpenXR

Since the `vtkRenderingOpenXR` API (and by extension the OpenXR specification ?) does not provide a way to get the state of the controllers & trackers:
* can the validity of the pose be inferred from the associated transform ?
* or do we need to improve `vtkRenderingOpenXR` API ?

To help answer, here are few observations:
- `VirtualReality.TrackerActive`, `VirtualReality.HMDActive` and `VirtualReality.ControllerActive` are set but not used (in `SlicerVirtualReality` extension)
- `VirtualReality.PoseStatus` is used in `qMRMLVirtualRealityTransformWidget::updateWidgetFromMRML` to exit if there are no change compared to the last status, or update the the icon representing the status of each trackers (including HMD and controllers)
- `VirtualReality.VRDeviceID` is set but not used (in `SlicerVirtualReality` extension)

Optionally, to improve the user experience, it would be nice to still have a way to report the status of each controllers & trackers.

Based on the observations, the critical information we need to have available through OpenXR are the controllers and trackers transforms.

cc: @LucasGandel @lassoan @cpinter @adamrankin

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the OpenVR transform API referenced in qMRMLVirtualRealityView_p.h and the HMD handling in qMRMLVirtualRealityView.cxx. Compare those paths with the available vtkRenderingOpenXR API, then inspect qMRMLVirtualRealityTransformWidget::updateWidgetFromMRML. Done means the required controller, HMD, and tracker transforms are available through OpenXR, with any supported status reporting defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-graphics
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.