isl-org / isl-org/Open3D

"project_to_rgbd_image" provides inconsistent depth and color images.

Open
#6,671 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### My Question

```python
device = o3d.core.Device('cuda:0')
pcd_tensor = o3d.t.geometry.PointCloud.from_legacy(pcd, device=device)
intrinisics = o3d.core.Tensor([[fx, 0, cx], [0, fy, cy], [0, 0, 1]], dtype=o3d.core.float32, device=device)

rgbd_reproj = pcd_tensor.project_to_rgbd_image(W, H, intrinisics)
fig, axs = plt.subplots(1, 2)
axs[0].imshow(np.asarray(rgbd_reproj.color.to_legacy()))
axs[1].imshow(np.asarray(rgbd_reproj.depth.to_legacy()))
```

The depth image and the color image provided by "project_to_rgbd_image" are inconsistent. The color image contains many black scattered invalid areas, while the depth image does not. I understand that the black areas in the color image represent areas where no point cloud is rendered, but why does the depth image have values in those areas?

![Figure_1](https://github.com/isl-org/Open3D/assets/131570332/bd371f2a-f2c8-4247-a29f-4110abf33981)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.