huggingface / huggingface/leLab

Feature Request: Support for OMX-AI in leLab

Open
#79 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
171
Forks
60
Avg merge
1d 52m
Merged PRs (30d)
5

Description

## Describe the Feature Request
Provide native support for the **OMX-AI** robotic arm in the `leLab` web GUI, alongside the current SO-100 series.

Currently, the underlying `lerobot` library officially supports Dynamixel motors and the OMX-AI platform. However, `leLab` restricts connection and usage to the hardcoded SO-101 leader/follower arms. Introducing abstraction for robot configurations will allow users of OMX-AI to also benefit from leLab's zero-code web UI for calibration, teleoperation, recording, and inference.

## Current Limitations / Code References
Here are the areas in the codebase that are currently hardcoded for SO-101:

### 1. Backend Configurations (`lelab`)
* **Teleoperation & Verification**: In `lelab/teleoperate.py` (lines 23–24), `SO101FollowerConfig`, `SO101Follower`, `SO101LeaderConfig`, and `SO101Leader` are directly imported. They are instantiated directly in lines 169–184.
* **Dataset Recording**: In `lelab/record.py` (lines 27–31), `SO101FollowerConfig` and `SO101LeaderConfig` are imported and instantiated on lines 159–166.
* **Calibration**: In `lelab/calibrate.py` (lines 256–263), calibration configuration setup specifically expects `SO101FollowerConfig` and `SO101LeaderConfig`.
* **Inference Rollouts**: In `lelab/rollout.py` (line 284), `--robot.type=so101_follower` is hardcoded as an argument for the rollout subprocess.

### 2. Frontend Calibration Configuration (`frontend`)
* **Joint Calibration Targets**: In `frontend/src/lib/calibrationTargets.ts` (lines 4–25), the target calibration limits are explicitly hardcoded for `SO101_LEADER_TARGETS` and `SO101_FOLLOWER_TARGETS` (e.g., `shoulder_pan`, `shoulder_lift`, `elbow_flex`, `wrist_flex`, `wrist_roll`, `gripper`).

---

## Proposed Solutions
1. **Dynamic Robot Configuration Loading**:
* Refactor the backend (e.g., `teleoperate.py`, `record.py`, `calibrate.py`) to dynamically import and build configuration classes depending on the target robot specified (e.g., loading `OMXAIFollowerConfig` / `OMXAILeaderConfig` if `omx-ai` is chosen).
2. **Abstract Calibration Targets on the Frontend**:
* Expose the calibration joint targets dynamically from the backend (by reading properties from the respective `lerobot` robot class) or refactor the frontend configuration to support mappings based on the active robot configuration profile.
3. **Parameterize Rollouts**:
* Allow the frontend or user configuration file to dynamically set the `--robot.type` parameter instead of hardcoding it in `rollout.py`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the SO-101-specific configuration paths in lelab/teleoperate.py, lelab/record.py, lelab/calibrate.py, and lelab/rollout.py, then inspect frontend/src/lib/calibrationTargets.ts. The work is complete when OMX-AI can be selected and supported across teleoperation, recording, calibration, inference rollouts, and frontend calibration targets.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
backend, frontend, robotics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.