NVIDIA / NVIDIA/IsaacTeleop

How to calibrate the controller-to-wrist transform for a Quest 3 controller mounted on a Manus glove?

Open
#658 2 comments 0 reactions 1 assignee View on GitHub

@sgrizan-nv is already working on this.

Since Jun 12, 2026.

Dominant language
Python
Stars
385
Forks
88
Avg merge
3d 23h
Merged PRs (30d)
82

Description

Title: How to calibrate Meta Quest 3 controller mounted on Manus glove to wrist frame?

Hi Isaac Teleop team,

I am trying to use a Meta Quest 3 controller together with a Manus glove. My setup is that the Quest 3 controller is physically mounted on the Manus glove, and I want to use the controller pose to drive the robot wrist/end-effector pose.

However, I am not sure how to calibrate the transform between:

  • the Meta Quest 3 controller tracking frame
  • the human wrist / Manus glove wrist frame
  • the robot end-effector control frame

From reading the code, it looks like some offsets are applied using a hard-coded transform matrix / hard-coded rotation and translation offsets. I understand the idea of applying an offset, but I do not know how that matrix should be generated for my own hardware setup.

Could you provide guidance on the recommended calibration procedure?

Specifically:

  1. How should I estimate the fixed transform from the mounted Quest controller frame to the wrist frame?
  2. Is there an existing calibration script or workflow in Isaac Teleop for this?
  3. Should the offset be defined in the controller local frame, the OpenXR/world frame, or the robot target frame?
  4. For Se3RetargeterConfig, how should I correctly set target_offset_roll, target_offset_pitch, target_offset_yaw, and target_offset_x/y/z for a custom controller-on-glove mount?
  5. Is the intended workflow to manually tune these values, or to record a reference pose and compute the transform automatically?

My goal is to use Manus glove data for hand/finger retargeting while using the mounted Quest controller pose as a stable wrist pose input for robot arm teleoperation.

Any explanation, example, or recommended calibration method would be very helpful.

Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.