isl-org / isl-org/Open3D

Non-blocking visualization on VoxelGrid fails to update visualization

Open
#2,047 10 comments 0 reactions 1 assignee Claimed by @yxlao View on GitHub
question visualization voxelgrid
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

__The issue__
I'm trying to convert realsense point cloud to voxel and visualize it in real time.

I followed this [example](https://github.com/intel-isl/Open3D/blob/master/examples/python/ReconstructionSystem/sensors/realsense_pcd_visualizer.py) to visualize realsense point cloud in real time. But instead of passing point cloud object to the visualizer, I passed a VoxelGrid object. The first VoxelGrid is displayed in the visualization window, but it is not getting updated at all. The same thing occurs when I passed a point cloud processed using `voxel_down_sample()`. Could it be that this is a bug of Visualizer class?

__To reproduce__
Here is part of my code:
```python
if __name__ == "__main__":
# Realsense initialization
pipeline = rs.pipeline()
config = rs.config()
config.enable_stream(rs.stream.depth, 1280, 720, rs.format.z16, 15)
config.enable_stream(rs.stream.color, 1280, 720, rs.format.rgb8, 15)

# Start streaming
profile = pipeline.start(config)
depth_sensor = profile.get_device().first_depth_sensor()
depth_sensor.set_option(rs.option.visual_preset, Preset.Default)
depth_scale = depth_sensor.get_depth_scale()

# We will not display the background of objects more than
# clipping_distance_in_meters meters away
clipping_distance_in_meters = 3 # 3 meter
clipping_distance = clipping_distance_in_meters / depth_scale

# Create an align object
align_to = rs.stream.color
align = rs.align(align_to)

vis = o3d.visualization.Visualizer()
vis.create_window()
vis.get_render_option().background_color = np.array([0, 0, 0])

pcd = o3d.geometry.PointCloud()
flip_transform = [[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]]

# Streaming loop
frame_count = 0
try:
while True:
dt0 = datetime.now()

# Get frameset of color and depth
frames = pipeline.wait_for_frames()

# Get aligned frames
aligned_frames = align.process(frames)
aligned_depth_frame = aligned_frames.get_depth_frame()
color_frame = aligned_frames.get_color_frame()
intrinsic = o3d.camera.PinholeCameraIntrinsic(
get_intrinsic_matrix(color_frame))

# Validate that both frames are valid
if not aligned_depth_frame or not color_frame:
continue

depth_image = o3d.geometry.Image(
np.array(aligned_depth_frame.get_data()))
color_temp = np.asarray(color_frame.get_data())
color_image = o3d.geometry.Image(color_temp)

rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
color_image,
depth_image,
depth_scale=1.0 / depth_scale,
depth_trunc=clipping_distance_in_meters,
convert_rgb_to_intensity=False)
temp = o3d.geometry.PointCloud.create_from_rgbd_image(
rgbd_image, intrinsic)
temp.transform(flip_transform)
pcd.points = temp.points
pcd.colors = temp.colors
#downpcd = pcd.voxel_down_sample(voxel_size=0.05)
voxel_grid = o3d.geometry.VoxelGrid.create_from_point_cloud(
pcd, voxel_size=0.05)
print(voxel_grid)

if frame_count == 0:
#vis.add_geometry(downpcd)
#vis.add_geometry(pcd)
vis.add_geometry(voxel_grid)

#vis.update_geometry(downpcd)
#vis.update_geometry(pcd)
vis.update_geometry(voxel_grid)
vis.poll_events()
vis.update_renderer()

process_time = datetime.now() - dt0
print("FPS: " + str(1 / process_time.total_seconds()))
frame_count += 1

finally:
pipeline.stop()
vis.destroy_window()
```

__Environment__
- OS: Ubuntu 18.04
- Python: 3.6.8
- Open3D: 0.8.0, 0.10.0

Thanks for any help you can provide.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.