isl-org / isl-org/Open3D

Tensorboard-plugin throws Error when selecting RGB-shader with custom vertex features

Open
#6,950 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### Describe the issue

In the tensorboard plugin, I want to visualize two different colorings ("vertex_colors") and ("vertex_modified_colors") of one object.

Visualizing the color stored in "vertex_colors" works, but visualizing a secondary color option does not.
It works to visualize one channel of the additional vertex attribute with the shader `Colormap (Rainbow)` and `Colormap (Greyscale)`, but `RGB` does not work

### Steps to reproduce the bug

```python
1. Generate the tensorboard logs with the code below
2. Open tensorboard
3. As Data, select `modified_colors`
4. Use Shader `RGB`
#####################################################################################

import open3d as o3d
import numpy as np

# Monkey-patch torch.utils.tensorboard.SummaryWriter
from open3d.visualization.tensorboard_plugin import summary
# Utility function to convert Open3D geometry to a dictionary format
from open3d.visualization.tensorboard_plugin.util import to_dict_batch
from torch.utils.tensorboard import SummaryWriter

ply_point_cloud = o3d.data.PLYPointCloud()
pcd = o3d.io.read_point_cloud(ply_point_cloud.path)

logdir = "demo_logs/pytorch/small_scale"
writer = SummaryWriter(logdir)

pcd_dict = to_dict_batch([pcd])

# ADD ADDITIONAL VERTEX ATTRIBUTE
pcd_dict["vertex_modified_colors"] = np.copy(pcd_dict["vertex_colors"])

writer.add_3d('obj', pcd_dict, step=1)
writer.add_3d('obj', pcd_dict, step=2)
```

### Error message

Open3D WARNING] Tensor point cloud colors must have DType of Float32 not UInt8. Converting.
[Open3D INFO] OnDataChannelMessage: {"messageId":3,"class_name":"tensorboard/update_rendering","window_uid_list":["window_1"],"tag":"obj","render_state":{"property":"modified_colors","index":"0","shader":"defaultUnlit","colormap":{},"range":[0,255]},"updated":["shader"]}. Error processing message: TypeError: __eq__(): incompatible function arguments. The following argument types are supported:
1. (self: open3d.cpu.pybind.core.Dtype, arg0: open3d.cpu.pybind.core.Dtype) -> bool

Invoked with: UInt8,

At:
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/util.py(407): _normalize
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/util.py(670): apply
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/plugin.py(625): _update_rendering

### Expected behavior

The expectation is that a colored point cloud should be shown. Instead, it crashes

### Open3D, Python and System information

```markdown
- Operating system: Ubuntu 22.04
- Python version: 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0]
- Open3D version: 0.18.0 ( also tested on 0.18.0+553ca86)
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.