NVIDIA / NVIDIA/IsaacTeleop

Control panel / command center for the teleoperator

Open
#607 0 comments 0 reactions 1 assignee View on GitHub

@yanziz-nvidia is already working on this.

Since Jun 2, 2026.

NVIDIA
Dominant language
Python
Stars
385
Forks
88
Avg merge
3d 23h
Merged PRs (30d)
82

Description

Proposed by @yanziz-nvidia

Background

  • After testing the G1 robot and doing the demo run-through, we identified the need for a server-side control hub to manage running processes, accessible from a headset, phone, tablet, or computer.
  • A NVIDIA internal implementaiton: [TBD add screenshot]

Goal

  • Provide a control center UI that lets a teleoperator manage/monitor server-side
  • processes and tune parameters at runtime. It should be application-agnostic
  • the controlled app should be swappable (e.g. IL, Isaac OS, etc.).

Requirements

  • Manage multiple server-side processes from a single UI
  • Tune parameters at runtime
  • Accessible across devices: in-headset, on phone, on laptop
  • Served locally
  • Swappable underlying application
  • Ship as part of the OSS offering so users can fork and customize with their own features

Technical approach

  • Server side: extend wss.py to add the control-panel backend (no need to rebuild from the ground up).
  • Client side: a separate UI for now; can be improved later or merged into the server-side UI.
  • Consider expanding the existing Isaac teleop client rather than starting fresh.
  • A working version is reportedly testable on Orin. An existing mock (Claude-designed) exists but needs real polish — design quality needs to improve.

Open questions

  • Keep the client UI separate, or merge it into the server-side UI?
  • Extend the Isaac teleop client, or build a dedicated panel?
  • Include a teleop camera view? (Many teleop portals from other companies include one.)
  • Primary runtime target — on the robot, on the VM running IL, or both?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.