isl-org / isl-org/Open3D

How to filter out weird background artifact in Poisson surface reconstruction?

Open
#6,142 10 comments 1 reaction 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

I wanted to write this question on the Open3D forum, but the activation mail for my account never arrived, checked my spam as well, and had it resent. Nothing..... I am using Azure Kinect cameras to create a face mesh, first I create a point cloud from the RGB-D image. Which I then apply the Poisson surface reconstruction to. However after the surface reconstruction this weird square like artifact shows up in the mesh:

![Mesh front view](https://github.com/isl-org/Open3D/assets/82316777/6359a20a-76b1-453b-a441-e145ce75a851)
![Mesh side view](https://github.com/isl-org/Open3D/assets/82316777/6c0c95c3-ada7-40f2-9415-01ed65797981)
In the point clouds this artifact doesn't show:

![Point cloud](https://github.com/isl-org/Open3D/assets/82316777/85e3e7a8-2993-4ae6-bc29-9c34cb377a43)
![Point cloud with normals](https://github.com/isl-org/Open3D/assets/82316777/a21d7dcc-5e28-46cd-8de5-3b6e05f52241)



I think there might be some sort of canvas defined that the poisson reconstruction tries to fill, but this doesn't show in the point clouds. Maybe cropping the point clouds might solve this, but as far as I am aware this would need manual user input, which isn't allowed for the application that I'm making. It has to be automated. Does anyone know how this problem can be solved?




The code I'm using:
```python
def depth_to_point_cloud():

# Load JSON files
with open('intrinsic1.json') as f:
intrinsic_json_1 = json.load(f)

test_img = imageio.v2.imread("color1.jpg")
height, width = test_img.shape[:2]

# Load color and depth images
col_img_1 = open3d.io.read_image("color1.jpg")
dep_img_1 = open3d.io.read_image("depth1.png")

# Create RGBD images
rgbd1 = open3d.geometry.RGBDImage.create_from_color_and_depth(col_img_1, dep_img_1, convert_rgb_to_intensity = False)

# Create pinhole cameras
phc = open3d.camera.PinholeCameraIntrinsic(width, height, intrinsic_json_1["intrinsic_matrix"][2], intrinsic_json_1["intrinsic_matrix"][3], intrinsic_json_1["intrinsic_matrix"][0], intrinsic_json_1["intrinsic_matrix"][1])

# Create point clouds
pcd = open3d.geometry.PointCloud.create_from_rgbd_image(rgbd1, phc)

pcd.transform([[1,0,0,0],[0,-1,0,0],[0,0,-1,0],[0,0,0,1]])
pcd.estimate_normals(search_param=open3d.geometry.KDTreeSearchParamHybrid(radius=0.1, max_nn=30))
pcd.orient_normals_consistent_tangent_plane(100)
mesh, _ = open3d.geometry.TriangleMesh.create_from_point_cloud_poisson(pcd, 9)
open3d.visualization.draw_geometries([mesh], mesh_show_back_face=True)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.