dimensionalOS / dimensionalOS/dimos
Monocular SLAM experiment
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.5k
- Forks
- 808
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 233
Description
let's try
odometry, loop closure
use orbslam3 for odometry + loop closure - write a native module, nix build - implemented @ https://github.com/dimensionalOS/dimos/pull/1598
dense pointclouds after odom is ok
use depthanything 3
not single image to depth! - this is trash noise, use temporal window, submit multiple frames with paralax - how do we deal with timestamp alignment? calculated depth needs to be ts aligned since it depends on odom
- use sharpness_barrier to select sharp frames
- calculate frames that are distinct enough (either optical flow or just pixel diff)
- submit blocks of frames to depth_anything in bulk
- feed extrinsics from orbslam3 into depth_anything (https://github.com/ByteDance-Seed/depth-anything-3/blob/main/docs/API.md#-pose-conditioned-depth-estimation)
- convert depth to pointclouds, build a global map - write a module
depth anything 3 module implemented @ https://github.com/dimensionalOS/dimos/pull/1597
https://www.youtube.com/watch?v=EX5d3sjTI0w
https://www.youtube.com/watch?v=349oLODVVtI
global map
use new memory2 to assemble frames based on global keypoint locations, as loop closure moves keypoints around, (each pointcloud frame placed in relation to it's nearest keypoint, then fused into global voxel grid)
global voxel grid then pruned with some statistical cutoff for hits, better sparse then wrong
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the odometry and loop-closure work in PR #1598 and the Depth Anything 3 work in PR #1597. The issue names no files, tests, or entry points, so trace those PRs first to determine what remains. Done would include the temporal depth workflow and a global voxel-map assembly with pruning as described here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, robotics
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100