dimensionalOS / dimensionalOS/dimos

Quest teleop Single Module

Open
#1,222 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

spec
Dominant language
Python
Stars
4.5k
Forks
808
Avg merge
3d 5h
Merged PRs (30d)
233

Description

reviewing https://github.com/dimensionalOS/dimos/pull/1215/ and looks ok, just there is a cleaner arch approach.

For now we have in one terminal a parallel TS system that collects raw pose/buttons data and publishes to LCM
quest -> typescript deno server -> LCM
these are not dimos modules, this is typescript publishing to LCM directly

And another terminal runs actual dimos
LCM -> questTeleopModule(s) -> Poses, Twists, etc

Result is that we have two parallel systems (TS and dimos), hard dependancy on LCM as transport between them, with blueprints not understanding topics used.

Ideally as a dimos user I want just
quest -> questTeletopModule -> Poses, Twists, etc

And I expect questTeletopModule to setup web server and whatever is required.

This is easily achiveable. Instead of sending fully encoded LCM packets (that include topics encoding and are passed directly to UDP) you just encode actual data (without a topic)

can read first of all about how LCM provides us with data encoding methods on all msgs https://github.com/dimensionalOS/dimos/blob/main/docs/concepts/lcm.md

from https://jsr.io/@dimos/msgs

// Construction with partial init
const pose = new geometry_msgs.Pose({
  position: new geometry_msgs.Point({ x: 1, y: 2, z: 3 }),
  orientation: new geometry_msgs.Quaternion({ w: 1 }),
});

// Encode to bytes (includes 8-byte hash prefix)
const bytes: Uint8Array = pose.encode();

send those bytes off

on the python side you'd start a simple webserver with websocket connection.
then when receive a msg,

Then you have 2 options

publish those bytes directly to a standard transport without decoding

(transport needs to accept LCM encoding, it uses from dimos.protocol.pubsub.encoders import LCMEncoderMixin)

self.left_pose.publish(received_bytes) (https://github.com/dimensionalOS/dimos/pull/1223 needs merge for this first)

This way you can have standard transports on your module, blueprints understand it etc, and it's very efficient since you are not decoding messages coming from quest, just passing them

decode & do whatever

this is actually better imo

left_pose = PoseStamped.lcm_decode(bytes)

and do whatever you wish with the pose object, convert to twist, do those follow up control transformations, publish when ready

left_pose = PoseStamped.lcm_decode(those_bytes)
self.left_twist.publish(left_pose.to_twist())

This way you don't have 2 dimos modules for quest (receiver/server + conversions for control like twist etc) but a single module

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading docs/concepts/lcm.md, the @dimos/msgs encoding example, and the transport work referenced in PR #1223. Trace the current Quest TypeScript-to-LCM path and questTeleopModule setup; done means one dimos module owns the web server, receives Quest data, and publishes or decodes it without the separate parallel system.

Written by the indexing model from the issue text.

Assessment

Tech stack
deno, python, typescript
Domain
backend-api-design, robotics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.