KitwareMedical / KitwareMedical/SlicerVirtualReality

OpenGL context issues when observing the VR camera

Open
#37 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
137
Forks
62
Avg merge
8h 8m
Merged PRs (30d)
8

Description

If the VR camera is observed, and the callback makes changes in the MRML scene, then the OpenGL context produces strange symptoms, such as showing nothing, or even a previously shown scene in one VR view while the other is showing the proper scene.

To reproduce:
* Add observation to VR camera
```
vrViewWidget = slicer.modules.virtualreality.viewWidget()
rendererCollection = vrViewWidget.renderWindow().GetRenderers()
vrRenderer = rendererCollection.GetItemAsObject(0)
self.vrCamera = vrRenderer.GetActiveCamera()
self.addObserver(self.vrCamera, vtk.vtkCommand.ModifiedEvent, self.onVrCameraModified)
```
* Set a matrix
```
def onVrCameraModified(self,caller,event):
avatarTransformNode = self.headAvatarModelNode.GetParentTransformNode()

# Move and orient avatar model to VR camera
import numpy as np
position = self.vrCamera.GetPosition()
positionVector = np.array(position)
focalPointVector = np.array(self.vrCamera.GetFocalPoint())
viewUpVector = np.array(self.vrCamera.GetViewUp())
# Z
zAxis = focalPointVector - positionVector
zAxisLength = np.linalg.norm(zAxis)
zAxis = zAxis / zAxisLength
# X
xAxis = np.array(viewUpVector)
# Y
yAxis = np.cross(zAxis, xAxis)
# Matrix
avatarTransformMatrix = vtk.vtkMatrix4x4()
for z in xrange(3):
avatarTransformMatrix.SetElement(z,2,zAxis[z])
for y in xrange(3):
avatarTransformMatrix.SetElement(y,1,yAxis[y])
for x in xrange(3):
avatarTransformMatrix.SetElement(x,0,xAxis[x])
for p in xrange(3):
avatarTransformMatrix.SetElement(p,3,position[p])
avatarTransformNode.SetMatrixTransformToParent(avatarTransformMatrix)
```

If a QTimer.singleShot triggers a function that does this calculation and SetMatrixTransformToParent, then everything works well.

Maybe if the VR camera is modified with a different timing, then this issue wouldn't come up. For example it could be done as the first step and not while doing the rendering, or an event could be invoked that sets the camera.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the observer path from vrViewWidget().renderWindow() through GetActiveCamera(), onVrCameraModified, and SetMatrixTransformToParent. Compare that behavior with the QTimer.singleShot version, and verify that both VR views continue showing the correct scene after the callback changes the MRML scene.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
ar-vr-xr, computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.