isl-org / isl-org/Open3D

Convert and visualise in realtime Realsense poincloud to Open3D pointcloud

Open
#7,379 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [x] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [x] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [x] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### My Question

I wish to convert and visualise in realtime Realsense poincloud to Open3D pointcloud. This I can easily do if I were to first save the realsense pointcloud as a .ply file and import this as a open3d pointcloud, but I don't know how to perform the direct conversion.

I've been looking at many different issues that have looked at this case (such as https://github.com/isl-org/Open3D/issues/473) and feel that the following code block should be the solution:

```
import pyrealsense2 as rs
import numpy as np
import open3d as o3d

# Pipeline & config
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

profile = pipeline.start(config)

profile = pipeline.get_active_profile()
depth_profile = rs.video_stream_profile(profile.get_stream(rs.stream.depth))
depth_intrinsics = depth_profile.get_intrinsics()
w, h = depth_intrinsics.width, depth_intrinsics.height

pc = rs.pointcloud()

try:
vis = o3d.visualization.Visualizer()
vis.create_window("RealSense + Open3D")
pcd = o3d.geometry.PointCloud()

while True:
vis.add_geometry(pcd)
pcd.clear()

frames = pipeline.wait_for_frames()
color = frames.get_color_frame()
depth = frames.get_depth_frame()
if not color or not depth:
continue

pc.map_to(color)
points = pc.calculate(depth)
vtx = np.asarray(points.get_vertices(2)).reshape(h, w, 3)
pcd.points = o3d.utility.Vector3dVector(vtx)
vis.update_geometry()
vis.poll_events()
vis.update_renderer()

finally:
pipeline.stop()
```

However, I received the following error from the line `pcd.points = o3d.utility.Vector3dVector(vtx)`:
```
RuntimeError: Unable to cast Python instance of type to C++ type
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.