[Bug Report] isaacsim.kit.xr.teleop.bridge extension not found - CloudXR teleoperation cannot fully function
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 385
- Forks
- 88
- Avg merge
- 3d 23h
- Merged PRs (30d)
- 82
Description
Describe the bug
Following the official CloudXR teleoperation documentation:
https://isaac-sim.github.io/IsaacLab/develop/source/how-to/cloudxr_teleoperation.html
The isaacsim.kit.xr.teleop.bridge extension cannot be resolved from any extension registry when using Isaac Sim 6.0 (pip install) with Isaac Teleop 1.0.191. This prevents CloudXR teleoperation from functioning - no controller input reaches Isaac Teleop from the VR headset.
Failed to resolve extension dependencies. Failure hints:
* No versions of isaacsim.kit.xr.teleop.bridge that satisfies: [enable list]
- Available packages for isaacsim.kit.xr.teleop.bridge version *: (none found)
Synced registries:
- kit/default : found 0 packages (couldn't connect or empty)
- kit/sdk : found 0 packages (couldn't connect or empty)
- kit/prod/default : found 931 packages
- kit/prod/sdk : found 785 packages
The kit/default registry (pdx.s8k.io) is unreachable (TLS handshake fails). The extension is also not included in the Docker image (nvcr.io/nvidia/isaac-sim:6.0.0-dev2).
Additionally, omni.kit.xr.system.openxr does not expose get_instance_proc_addr(), which the bridge extension is supposed to polyfill (session_lifecycle.py line 68-82). Without the polyfill, _acquire_kit_oxr_handles() crashes with AttributeError.
What works
- G1 simulation rendering streamed to PICO 4 Ultra Enterprise via CloudXR
- Camera view follows head movement in the headset
What does not work
- Controller input from PICO does not reach Isaac Teleop at all (no pose tracking, no button input from headset controllers)
ControllerTracker update failedrepeats continuouslyXR_ERROR_FORM_FACTOR_UNSUPPORTED: xrGetSystem(getInfo->formFactor) no matching system (given: 2, first: 1)- Kit XR requests form factor 2 (handheld/AR) but CloudXR Monado only supports form factor 1 (HMD)
Steps to reproduce
- Install Isaac Sim 6.0 via pip:
pip install "isaacsim[all,extscache]==6.0.0" - Install Isaac Lab (develop branch)
- Install Isaac Teleop:
pip install "isaacteleop[retargeters,ui,cloudxr]~=1.0.0" --extra-index-url https://pypi.nvidia.com - Start CloudXR:
python -m isaacteleop.cloudxr --accept-eula(withNV_CXR_ENABLE_PUSH_DEVICES=0) - Launch teleop:
./isaaclab.sh -p scripts/environments/teleoperation/teleop_se3_agent.py --task Isaac-PickPlace-Locomanipulation-G1-Abs-v0 --visualizer kit --xr - Click Start XR in the XR tab, connect PICO via CloudXR.js client
Questions
- Is the
isaacsim.kit.xr.teleop.bridgeextension supposed to be included in the Isaac Sim 6.0 pip package or downloaded from a registry? - Is there a workaround to get controller input working without the bridge extension?
- Is this expected to work with Isaac Sim 6.0 Early Developer Release, or should we wait for the GA release?
Related issue: https://github.com/isaac-sim/IsaacLab/issues/5221
System Info
- OS: Ubuntu 22.04
- GPU: NVIDIA RTX PRO 6000 Blackwell (98GB VRAM)
- Driver: 580.126.09
- Isaac Sim: 6.0.0 (pip)
- Isaac Lab: develop branch
- Isaac Teleop: 1.0.191
- CloudXR: 6.1.0
- VR Headset: PICO 4 Ultra Enterprise
- Python: 3.12.13
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the pip installation and CloudXR launch sequence, then inspect session_lifecycle.py lines 68-82 and the teleop_se3_agent.py entry point. Check whether isaacsim.kit.xr.teleop.bridge is packaged or available through the listed registries, and verify that the extension resolves without the get_instance_proc_addr() failure. Done means controller input reaches Isaac Teleop without repeated tracker errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ar-vr-xr
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100