dimensionalOS / dimensionalOS/dimos

Teleop restructuring

Open
#1,113 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

spec
Dominant language
Python
Stars
4.5k
Forks
808
Avg merge
3d 5h
Merged PRs (30d)
233

Description

Teleop Device Categories

teleop stack divided into device categories, each with its own base module:

  1. VR Headsets (Browser-based WebXR)
    Meta Quest
    Apple Vision Pro
    PICO
  2. Smart Devices (Browser-based / Native App)
    iPhone IMU
    Smart watches (Maybe)
  3. Hand Controllers (USB/Bluetooth)
    Joysticks (single/dual stick)
    Game controllers (Xbox, PlayStation)
    SpaceMouse
    Keyboard

Design Rationale: Each category has distinct input modalities (6DoF tracking vs IMU vs discrete axes), so a single base class for all devices would be bloated. Instead, each category has its own base module (VRBaseModule, SmartDeviceBaseModule, HandControllerBaseModule) with shared interfaces where appropriate.

class VRBaseModule(Module):
    """Base for all VR headset teleop devices."""
    
    # Lifecycle
    def start(self) -> None: ...
    def stop(self) -> None: ...
    
    # Teleop session (toggled by button press)
    def start_teleop(self) -> None: ...
    def stop_teleop(self) -> None: ...
    
    # Core logic
    def calibrate(self) -> bool:
        """Reset t0, store P0/R0 as reference. Called on teleop start."""
    
    def compute_delta(
        self,
        controller_poses: list[PoseStamped | None],
    ) -> list[PoseStamped | None]:
        """Compute delta from calibrated reference. Always PoseStamped."""
    
    def convert_to_output(
        self, 
        pose: PoseStamped, 
        output_type: type
    ) -> PoseStamped | TwistStamped:
        """Convert to target output type at publish boundary."""

class QuestModule(VRBaseModule):
    """Meta Quest teleop via WebSocket/LCM bridge."""
    
    # Inputs (from Deno bridge)
    left_transform: In[LCMTransform]
    right_transform: In[LCMTransform]
    left_trigger: In[Float32]
    right_trigger: In[Float32]
    teleop_enable: In[Bool]  # X button
    
    # Outputs (user configurable - PoseStamped or TwistStamped)
    left_controller: Out[PoseStamped] 
    right_controller: Out[PoseStamped]
    left_trigger_value: Out[Float32]
    right_trigger_value: Out[Float32]
    
    # Callbacks
    def _on_teleop_enable(self, msg: Bool) -> None:
        """Toggle start_teleop/stop_teleop on button press."""
    def _on_transform(self, index: int, transform: Transform) -> None: ...
    def _on_trigger(self, index: int, msg: Float32) -> None: ...
    
    # Main loop
    def control_loop(self) -> None:
        """Compute deltas, convert to output type, publish."""

  • X press to start, and X press to stop. When pressed X again, it starts from zero

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points. Start by locating the existing teleoperation modules and Module implementations, then compare them with the proposed VR, smart-device, and hand-controller interfaces. Done means the category structure and Quest behavior, including X-button start/stop toggling and calibrated outputs, are implemented and verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
robotics
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.