KitwareMedical / KitwareMedical/SlicerVirtualReality
OpenGL context issues when observing the VR camera
- Dominant language
- C++
- Stars
- 137
- Forks
- 62
- Avg merge
- 8h 8m
- Merged PRs (30d)
- 8
Description
If the VR camera is observed, and the callback makes changes in the MRML scene, then the OpenGL context produces strange symptoms, such as showing nothing, or even a previously shown scene in one VR view while the other is showing the proper scene.
To reproduce:
* Add observation to VR camera
```
vrViewWidget = slicer.modules.virtualreality.viewWidget()
rendererCollection = vrViewWidget.renderWindow().GetRenderers()
vrRenderer = rendererCollection.GetItemAsObject(0)
self.vrCamera = vrRenderer.GetActiveCamera()
self.addObserver(self.vrCamera, vtk.vtkCommand.ModifiedEvent, self.onVrCameraModified)
```
* Set a matrix
```
def onVrCameraModified(self,caller,event):
avatarTransformNode = self.headAvatarModelNode.GetParentTransformNode()
# Move and orient avatar model to VR camera
import numpy as np
position = self.vrCamera.GetPosition()
positionVector = np.array(position)
focalPointVector = np.array(self.vrCamera.GetFocalPoint())
viewUpVector = np.array(self.vrCamera.GetViewUp())
# Z
zAxis = focalPointVector - positionVector
zAxisLength = np.linalg.norm(zAxis)
zAxis = zAxis / zAxisLength
# X
xAxis = np.array(viewUpVector)
# Y
yAxis = np.cross(zAxis, xAxis)
# Matrix
avatarTransformMatrix = vtk.vtkMatrix4x4()
for z in xrange(3):
avatarTransformMatrix.SetElement(z,2,zAxis[z])
for y in xrange(3):
avatarTransformMatrix.SetElement(y,1,yAxis[y])
for x in xrange(3):
avatarTransformMatrix.SetElement(x,0,xAxis[x])
for p in xrange(3):
avatarTransformMatrix.SetElement(p,3,position[p])
avatarTransformNode.SetMatrixTransformToParent(avatarTransformMatrix)
```
If a QTimer.singleShot triggers a function that does this calculation and SetMatrixTransformToParent, then everything works well.
Maybe if the VR camera is modified with a different timing, then this issue wouldn't come up. For example it could be done as the first step and not while doing the rendering, or an event could be invoked that sets the camera.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the observer path from vrViewWidget().renderWindow() through GetActiveCamera(), onVrCameraModified, and SetMatrixTransformToParent. Compare that behavior with the QTimer.singleShot version, and verify that both VR views continue showing the correct scene after the callback changes the MRML scene.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- ar-vr-xr, computer-graphics
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100