dimensionalOS / dimensionalOS/dimos
Tooling to send arm urdf to rerun
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.5k
- Forks
- 808
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 233
Description
Summary
Add URDF-based arm visualization to Rerun so manipulation can be visualized the same way navigation uses Rerun for maps — without depending on Drake/Meshcat.
Context
Currently, all manipulation visualization goes through Drake's Meshcat server (ManipulationModule with enable_viz=True). This is a separate browser window, disconnected from the rest of the dimOS observability stack. Meanwhile, navigation already visualizes maps, poses, and sensor data in Rerun — the standard dimOS visualization tool.
The goal is to load a robot arm URDF into Rerun's 3D view and update joint states in real time, so keyboard teleop, planning, and general manipulation can be visualized alongside camera feeds, point clouds, and other sensor data in a single Rerun session.
Key pointers
| What | Where |
|---|---|
| Rerun SDK docs for URDF | Rerun supports rr.log("robot", rr.Asset3D(...)) and rr.Transform3D per link |
| Current Drake visualization | dimos/manipulation/manipulation_module.py — enable_viz flag |
| Keyboard teleop blueprints (use Meshcat) | dimos/robot/manipulators/xarm/blueprints.py |
| Joint state message | dimos/msgs/sensor_msgs/JointState.py |
| URDF locations | data/xarm_description/, data/openarm_description/, etc. |
| Rerun integration examples | dimos/navigation/ — see how nav logs to Rerun |
| Coordinator publishes joint state | ControlCoordinator with publish_joint_state=True |
Deliverables
- Utility/module that loads a URDF and logs the kinematic tree to Rerun (links as meshes, joints as
rr.Transform3D) - Real-time joint state updates — subscribe to
JointStatefrom the coordinator and update link transforms each tick - Works as a drop-in replacement for
ManipulationModule's Drake viz in keyboard teleop blueprints - At least one blueprint (e.g.
keyboard_teleop_xarm7) running with Rerun visualization instead of Meshcat - Verify it works with at least two different URDFs (e.g. xArm7 and OpenArm)
Notes
- Rerun has native support for mesh assets and hierarchical transforms — the approach is to log each URDF link as a mesh under a transform hierarchy and update joint transforms from FK.
- You'll need forward kinematics to map
JointState→ per-link transforms. Pinocchio is already a dependency and can do this, or you can parse the URDF joint tree directly. - The coordinator already publishes
JointStateover LCM whenpublish_joint_state=True— the visualizer just needs to subscribe. - This should not remove Drake/Meshcat support — just provide a Rerun alternative that can be toggled.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading dimos/manipulation/manipulation_module.py, dimos/robot/manipulators/xarm/blueprints.py, and the Rerun integration in dimos/navigation/. Trace JointState from ControlCoordinator and inspect the URDFs under data/xarm_description/ and data/openarm_description/. Done means a Rerun-based alternative supports live joint updates, works in a keyboard teleop blueprint, and is verified with two URDFs without removing Meshcat support.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability-sre, robotics
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100