NVIDIA / NVIDIA/IsaacTeleop

Migrate Isaac Lab / Sim to use OVRTX

Open
#622 0 comments 0 reactions 1 assignee View on GitHub

@hougantc-nvda is already working on this.

Since Jun 12, 2026.

NVIDIA
Dominant language
Python
Stars
385
Forks
88
Avg merge
3d 23h
Merged PRs (30d)
82

Description

Background: Omniverse Kit feature development ends after v110. Sim Teleop sits
at the top of a dependency chain that bottoms out on Kit — Sim Teleop is built
on Isaac Lab, Isaac Lab on Isaac Sim, Isaac Sim on Kit — once Kit stops
evolving, the XR teleoperation stack stops evolving with it. Isaac Lab has
already begun migrating to the kitless, Omniverse-libraries-only path; Sim
Teleop must follow to stay viable.

The gap: The kitless path lacks XR support. It requires an OpenXR-compliant
renderer that integrates with CloudXR for headset streaming. Such a library,
built on the OVRTX renderer SDK, is under development by the rendering
team.

This initiative: Integrate that library into Sim Teleop's kitless path. Isaac
Lab creates and drives the XR cameras and render products, with the render
loop coupled to the physics step; IsaacTeleop consumes operator
head/hand/controller input on the shared OpenXR session; a kitless launcher
manages the CloudXR runtime and session lifecycle end-to-end. Done means an
operator in a headset teleoperates a robot in Isaac Lab at parity with the
Kit-based XR path, with no Kit dependency in the loop.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.