isl-org / isl-org/Open3D

3D scene reconstruction

Open
#7,051 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### My Question

I'm using a intel realsense D455 to acquire depth images, turning them into point clouds and then using ICP to do scene reconstruction. I'm using python 3.11.7, open 3D 0.18.0. When performing ICP, not only the code is not working, it is crashing with no apparent error.

My current code is:

import numpy as np
import open3d as o3d

if __name__ == "__main__":

print("1. Load two point clouds and show initial pose")
source = o3d.io.read_point_cloud(".../testclouda.ply")
target = o3d.io.read_point_cloud(".../testcloudb.ply")

# Scale down points by dividing by 100 (assuming depth units in cm)
source.points = o3d.utility.Vector3dVector(np.asarray(source.points) / 100.0)
target.points = o3d.utility.Vector3dVector(np.asarray(target.points) / 100.0)

# Estimate normals
source.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.05, max_nn=30))
target.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.05, max_nn=30))

# Draw initial alignment
current_transformation = np.identity(4)

# Step 1: Point-to-point ICP Registration
print("2. Performing point-to-point ICP registration with a distance threshold of 0.02")
result_icp = o3d.pipelines.registration.registration_icp(source, target, 0.02 , current_transformation,o3d.pipelines.registration.TransformationEstimationPointToPoint())
current_transformation = result_icp.transformation

# Step 2: Point-to-plane ICP Registration
print("3. Performing point-to-plane ICP registration with a distance threshold of 0.02")
result_icp = o3d.pipelines.registration.registration_icp(
source, target, 0.02, current_transformation,
o3d.pipelines.registration.TransformationEstimationPointToPlane()
)
current_transformation = result_icp.transformation

# Step 3: Colored Point Cloud Registration
voxel_radius = [0.04, 0.02, 0.01]
max_iter = [50, 30, 14]
print("4. Performing multi-scale colored point cloud registration")

for scale in range(3):
iter_count = max_iter[scale]
radius = voxel_radius[scale]
print(f"Scale {scale+1} - Voxel size: {radius}, Iterations: {iter_count}")

# Downsample with voxel size
source_down = source.voxel_down_sample(radius)
target_down = target.voxel_down_sample(radius)

# Estimate normals for downsampled point clouds
source_down.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=radius * 2, max_nn=30))
target_down.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=radius * 2, max_nn=30))

# Apply colored point cloud registration
result_icp = o3d.pipelines.registration.registration_colored_icp(
source_down, target_down, radius, current_transformation,
o3d.pipelines.registration.TransformationEstimationForColoredICP(),
o3d.pipelines.registration.ICPConvergenceCriteria(
relative_fitness=1e-6, relative_rmse=1e-6, max_iteration=iter_count
)
)
current_transformation = result_icp.transformation
print(result_icp)

# Display the final registration result
# Create a deep copy of the source to apply the final transformation for visualization
source_temp = source.voxel_down_sample(0.02) # Downsample for better visualization if necessary
source_temp.transform(current_transformation)
o3d.visualization.draw_geometries([source_temp, target], window_name="Final Point Cloud Alignment")

However, in step 2, the code stops with the function:

o3d.pipelines.registration.registration_icp

Running the debugger shows:

Server[1] disconnected unexpectedly
Server[pid=2212] disconnected unexpectedly

The same problem also happens with other functions like: draw_registration_result_original_color.

Not sure what is the problem, any help or sugestions to better do scene reconstruction would be appreciated.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.