isl-org / isl-org/Open3D

Connot get same speed in c++ as in python

Open
#5,576 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

Hi.
I'm working on a project where Open3D is highly used. I started by writing the entire code in python to easily test different approaches. However, I have now decided that it is time to write the code in c++, as this is the desire at my workplace. I have managed to implement the same functionality in c++ but it seems like the code is now running much slower than in python. For some specific functions it seems almost 10x slower.

One of the parts much slower in c++ is the volume.integrate() function. Here is a snippet from my python project.

`

def useOpen3D(rob_cam1, rob_cam2, rob_cam3, pipeline_1, pipeline_2, pipeline_3):

volume = o3d.pipelines.integration.ScalableTSDFVolume(
voxel_length=4.0 / 512.0, sdf_trunc=0.02,
color_type=o3d.pipelines.integration.TSDFVolumeColorType.RGB8)

volume2 = o3d.pipelines.integration.ScalableTSDFVolume(
voxel_length=4.0 / 512.0, sdf_trunc=0.02,
color_type=o3d.pipelines.integration.TSDFVolumeColorType.RGB8)

volume3 = o3d.pipelines.integration.ScalableTSDFVolume(
voxel_length=4.0 / 512.0, sdf_trunc=0.01,
color_type=o3d.pipelines.integration.TSDFVolumeColorType.RGB8)

align = rs.align(rs.stream.color)

frameCounter = 0
while True:
profile, depth_frame, color_frame = getCamFrames(pipeline_1, align)
profile2, depth_frame2, color_frame2 = getCamFrames(pipeline_2, align)
profile3, depth_frame3, color_frame3 = getCamFrames(pipeline_3, align)

if not depth_frame or not color_frame or not depth_frame2 or not color_frame2 or not depth_frame3 or not color_frame3:
continue

img_color, img_depth = imgToNumpy(depth_frame, color_frame)
img_color2, img_depth2 = imgToNumpy(depth_frame2, color_frame2)
img_color3, img_depth3 = imgToNumpy(depth_frame3, color_frame3)

rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(img_color, img_depth, convert_rgb_to_intensity=False, depth_trunc=1.2)
rgbd2 = o3d.geometry.RGBDImage.create_from_color_and_depth(img_color2, img_depth2, convert_rgb_to_intensity=False, depth_trunc=1.2)
rgbd3 = o3d.geometry.RGBDImage.create_from_color_and_depth(img_color3, img_depth3, convert_rgb_to_intensity=False, depth_trunc=1.6)

intrinsics, pinhole_camera_intrinsic = getCamInfo(profile)
intrinsics2, pinhole_camera_intrinsic2 = getCamInfo(profile2)
intrinsics3, pinhole_camera_intrinsic3 = getCamInfo(profile3)

volume.integrate(rgbd, pinhole_camera_intrinsic, rob_cam1)
volume2.integrate(rgbd2, pinhole_camera_intrinsic2, rob_cam2)
volume3.integrate(rgbd3, pinhole_camera_intrinsic3, rob_cam3)

if frameCounter == 10:
volume.reset()
volume2.reset()
volume3.reset()

frameCounter += 1

if frameCounter == 30:
GenerateCroppedMesh(volume, volume2, volume3)
pipeline_1.stop()
pipeline_2.stop()
pipeline_3.stop()
break

`

The same functionality implemented in c++:

`
void open3dStuff(const sensor_data& sensor_1, const sensor_data& sensor_2, const sensor_data& sensor_3,
cv::Mat trans_501, cv::Mat trans_309, cv::Mat trans_699, open3d::camera::PinholeCameraIntrinsic intrics_1,
open3d::camera::PinholeCameraIntrinsic intrics_2, open3d::camera::PinholeCameraIntrinsic intrics_3) {

open3d::pipelines::integration::ScalableTSDFVolume* volume1 = new open3d::pipelines::integration::ScalableTSDFVolume(
4.0 / 512.0, 0.02, open3d::pipelines::integration::TSDFVolumeColorType::RGB8);

open3d::pipelines::integration::ScalableTSDFVolume* volume2 = new open3d::pipelines::integration::ScalableTSDFVolume(
4.0 / 512.0, 0.02, open3d::pipelines::integration::TSDFVolumeColorType::RGB8);

open3d::pipelines::integration::ScalableTSDFVolume* volume3 = new open3d::pipelines::integration::ScalableTSDFVolume(
4.0 / 512.0, 0.01, open3d::pipelines::integration::TSDFVolumeColorType::RGB8);

using legacyRGBDImage = open3d::t::geometry::RGBDImage;
using legacyImage = open3d::geometry::RGBDImage;

Eigen::Matrix eigen_501;
Eigen::Matrix eigen_309;
Eigen::Matrix eigen_699;

cv::cv2eigen(trans_501, eigen_501);
cv::cv2eigen(trans_309, eigen_309);
cv::cv2eigen(trans_699, eigen_699);

int counter = 0;
while (counter < 10)
{
legacyRGBDImage rgbd_1 = update_data(sensor_1.depth_data, sensor_1.color_data);
legacyRGBDImage rgbd_2 = update_data(sensor_2.depth_data, sensor_2.color_data);
legacyRGBDImage rgbd_3 = update_data(sensor_3.depth_data, sensor_3.color_data);

if (rgbd_1.IsEmpty() || rgbd_2.IsEmpty() || rgbd_3.IsEmpty())
{
continue;
}

std::cout << counter << "\n" << std::endl;

legacyImage im_1 = rgbd_1.ToLegacy();
legacyImage im_2 = rgbd_2.ToLegacy();
legacyImage im_3 = rgbd_3.ToLegacy();

std::shared_ptr o3d_rgbd_1 = open3d::geometry::RGBDImage::CreateFromColorAndDepth(im_1.color_, im_1.depth_, 1000.0, 1.2, false);
std::shared_ptr o3d_rgbd_2 = open3d::geometry::RGBDImage::CreateFromColorAndDepth(im_2.color_, im_2.depth_, 1000.0, 1.2, false);
std::shared_ptr o3d_rgbd_3 = open3d::geometry::RGBDImage::CreateFromColorAndDepth(im_3.color_, im_3.depth_, 1000.0, 1.6, false);

if (counter == 5) {
volume1->Reset();
volume2->Reset();
volume3->Reset();
}

volume1->Integrate(*o3d_rgbd_1, intrics_1, eigen_501);
volume2->Integrate(*o3d_rgbd_2, intrics_2, eigen_309);
volume3->Integrate(*o3d_rgbd_3, intrics_3, eigen_699);

counter++;
}
}
`

I have made sure to use same framerate, resolution and so on, for both codes.

Later on I also use the RegistrationICP() function, and here there seems to be a huge difference in running time again.
I anyone knows something, please let me know :)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.