NVIDIA / NVIDIA/IsaacTeleop

Surface pose validity (GRIP_IS_VALID) in deviceio / provide a stock PoseGate node for occlusion snap-back

Open
#731 0 comments 0 reactions 1 assignee View on GitHub

@jiwenc-nv is already working on this.

Since Jul 22, 2026.

Blocked
Dominant language
Python
Stars
385
Forks
88
Avg merge
3d 23h
Merged PRs (30d)
82

Description

Context

When XR controllers leave the headset's camera view, the OpenXR runtime keeps streaming an IMU-extrapolated pose and is_tracking remains true. On re-acquisition the pose teleports. Downstream, the robot lurches toward the ghost target.

Evidence

  • Measured lurches of up to 73 cm/frame on controller re-acquisition (Quest 3 via CloudXR, SO-101 follower).
  • We implemented a PoseGate at the example layer consuming the runtime's GRIP_IS_VALID flag; verified via synthetic replay: 73.2 → 0.2 cm/frame.
  • A rate limiter alone is the wrong tool here: it converts the teleport into a fast-but-legal slew toward a bogus target instead of rejecting it (this motivated the anomaly-rejection tier in PR #727).

Proposal

Either (in order of preference):

  1. Surface pose validity as a first-class deviceio output (GRIP_IS_VALID and friends), so any retargeting graph can gate on it, and/or
  2. Ship a stock PoseGate retargeter node that holds the last valid pose (and optionally signals the clutch to disengage) while validity is false.

This affects every XR consumer of the library, not just our integration — occlusion is routine in tabletop teleop (the operator's own body blocks the cameras constantly).

Happy to upstream our PoseGate implementation if the design is agreeable.


Part of the field-integration feedback series (SO-101 + reBot DevArm, June 29 – July 6). Priority suggestion: P0.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.