isl-org / isl-org/Open3D

Additional Features for void open3d::t::geometry::TSDFVoxelGrid::Integrate () function

Open
#4,104 2 comments 1 reaction 1 assignee Claimed by @theNded View on GitHub
feature request reconstruction
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

**Is your feature request related to a problem? Please describe.**
Instead of using an RGBD camera, my pipeline consists of an Android application using Unity + Googles AR-Core to create RGBD Data without any depth sensor (https://developers.google.com/ar/develop/unity-arf/depth/introduction). The data is sent via LAN to a PC running a C++ application utilizing Open3D. There are currently two major issues with this approach:

1. Android RGB camera has a much higher resolution compared to the depth map calculated (which is only 160x90 in my case). So at the moment I have to scale down the RGB image (obviously upscaling/interpolation of the depth image would also be possible)
2. The calculated Depth Data is quite noisy. The algorithm to integrate is able to smooth out quite a lot of the noise (slow-camera movement, correct TSDFVoxelGrid parameters,...) but it is far from perfect.

**Describe the solution you'd like**
1. Allow different resolutions of RGB image and Depth image for the integration function.
2. In one of the newer AR-Core version, there is the possibility to get Raw Depth data in combination with a confidence map instead of the full depth image (info here: https://developers.google.com/ar/develop/unity-arf/depth/raw-depth). The option to use this data in the integration algorithm (discarding depth pixels with low confidence) would increase the accuracy of the algorithm quite a lot I guess.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.