facebookresearch / facebookresearch/ocean

How to align the coordinate systems when using an external camera with the Quest headset?

Open
#49 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
790
Forks
94
PR merge metrics
No merged PRs in 30d

Description

Since the Quest does not allow users to access the built-in camera data, is there any official calibration procedure/algorithm to align the coordinate systems of the external camera and Quest's camera?

I didn't find any relevant information on this in the development documentation (https://facebookresearch.github.io/ocean/docs/introduction/).

While there are ways to perform the calibration process ourselves, for instance using a planar calibration checkerboard ([tutorial 1](https://medium.com/@chaitalibh.cb/camera-calibration-with-checkerboard-54f93af742a0), [tutorial 2](https://github.com/hjoonpark/MultiCamCalib), [tutorial 3](https://www.youtube.com/watch?v=EWqqseIjVqM&ab_channel=LearnOpenCV), [tutorial 4](https://www.youtube.com/watch?v=Wcnb197g2i0&ab_channel=KevinWood%7CRobotics%26AI), [tutorial 5](https://github.com/cvlab-epfl/multiview_calib), [stereo camera calibration explanation](https://www.youtube.com/watch?v=hUVyDabn1Mg&ab_channel=FirstPrinciplesofComputerVision)), it would be easier if there were an official solution from Meta.

A way to circumvent this calibration issue is if we can have access the Quest's onboard cameras and sensors. Is there any update on when Meta will release access to onboard the camera (and depth sensor)?

Note, though this has been [asked before](https://github.com/facebookresearch/ocean/issues/14), I am asking again as there wasn't any official solution/answer and that issue is now closed.

Thank you.

Contributor guide

Open the contributing guide

Research direction

Read the linked development documentation and prior issue #14 first; the payload names no source file, entry point, or test. Done would require an official answer or documentation covering external-camera calibration and the status of Quest onboard camera and depth-sensor access.

Written by the indexing model from the issue text.

Assessment

Domain
ar-vr-xr, computer-vision
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.