webarkit / webarkit/webarkit-testing
3D pose renders behind the camera (CV vs GL convention); matrixGL_RH throws
@kalwalt is already working on this.
Since May 29, 2026.
- Dominant language
- JavaScript
- Stars
- 2
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Summary
When a marker is successfully tracked, the getMarker event fires with a valid pose (real rotation + translation), but three.js examples still show no 3D object. The object is placed behind the camera and gets frustum-clipped.
Surfaced while implementing #30 (static-image Teblid example): the marker tracks correctly (Marker tracked ! Num. matches : 1000) and a found pose arrives, yet no sphere is visible.
Evidence (runtime)
Sample pose delivered to the scene (three.js column-major elements):
translation = (0.953, -0.384, +5.824)
Projection matrix from the same run (column-major):
[-1.785, 0, 0, 0, 0, 2.380, 0, 0, 0, 0, -1.0002, -1, 0, 0, -0.20002, 0]
This is a standard OpenGL projection (proj[11] = -1, camera looks down −Z). Plugging the object origin (0.953, -0.384, 5.824, 1) through it:
clip.w = -view.z = -5.824(negative → behind camera)ndc.z ≈ 1.03(> 1 → beyond far plane)
So the object is clipped and never rasterized, despite correct tracking.
Root cause
The pose field (from getPoseMatrix() → transMatToGLMat()) is in OpenCV camera convention (object in front ⇒ +Z), which is incompatible with the OpenGL projection used for rendering (object in front ⇒ −Z). The right-handed GL matrix matrixGL_RH (arglCameraViewRHf) — the one that should be fed to three.js — currently throws an exception (see recent dev commits experimenting with the pose matrix).
Suggested direction
- Provide a correct GL modelview/pose matrix (apply the full CV→GL handedness flip: negate Y and Z appropriately on both rotation and translation), or fix
arglCameraViewRHf/matrixGL_RHso it no longer throws and can be used directly. - Quick verification: negating the translation Z on the
posebeforesetMatrix(root.matrix, ...)should bring the object into view — useful as a confirmation while the proper conversion is sorted out.
Notes
- The fix likely lives in the
emscripten/WebARKitLibsubmodule (pose conversion:invertPose/cvToGl/arglCameraViewRHf) and would require rebuildingdist/WebARKit.js. - Affects all three.js worker examples, not just the static-image one from #30.
- Tracking itself is working; this is purely a pose-convention / rendering issue.
Related latent bug (already worked around in #30)
WebARKitTracker::processFrame calls GetTrackedFeaturesWarped() when a marker is detected on the very first frame (_frameCount == 0), before GetInitialFeatures() has populated the tracked-feature set. perspectiveTransform then runs on an empty point set and throws (CV_Assert(scn + 1 == m.cols)). The #30 static example works around this by feeding one blank warmup frame, but the library should guard the pose block against an empty feature set.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.