dimensionalOS / dimensionalOS/dimos

Monocular SLAM experiment

Open
#1,596 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

navigation perception research
Dominant language
Python
Stars
4.5k
Forks
808
Avg merge
3d 5h
Merged PRs (30d)
233

Description

let's try

odometry, loop closure

use orbslam3 for odometry + loop closure - write a native module, nix build - implemented @ https://github.com/dimensionalOS/dimos/pull/1598

dense pointclouds after odom is ok

use depthanything 3

not single image to depth! - this is trash noise, use temporal window, submit multiple frames with paralax - how do we deal with timestamp alignment? calculated depth needs to be ts aligned since it depends on odom

  1. use sharpness_barrier to select sharp frames
  2. calculate frames that are distinct enough (either optical flow or just pixel diff)
  3. submit blocks of frames to depth_anything in bulk
  4. feed extrinsics from orbslam3 into depth_anything (https://github.com/ByteDance-Seed/depth-anything-3/blob/main/docs/API.md#-pose-conditioned-depth-estimation)
  5. convert depth to pointclouds, build a global map - write a module
    depth anything 3 module implemented @ https://github.com/dimensionalOS/dimos/pull/1597

https://www.youtube.com/watch?v=EX5d3sjTI0w
https://www.youtube.com/watch?v=349oLODVVtI

global map

use new memory2 to assemble frames based on global keypoint locations, as loop closure moves keypoints around, (each pointcloud frame placed in relation to it's nearest keypoint, then fused into global voxel grid)

global voxel grid then pruned with some statistical cutoff for hits, better sparse then wrong

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the odometry and loop-closure work in PR #1598 and the Depth Anything 3 work in PR #1597. The issue names no files, tests, or entry points, so trace those PRs first to determine what remains. Done would include the temporal depth workflow and a global voxel-map assembly with pruning as described here.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, robotics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.