Tensorboard-plugin throws Error when selecting RGB-shader with custom vertex features
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
### Checklist
- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).
### Describe the issue
In the tensorboard plugin, I want to visualize two different colorings ("vertex_colors") and ("vertex_modified_colors") of one object.
Visualizing the color stored in "vertex_colors" works, but visualizing a secondary color option does not.
It works to visualize one channel of the additional vertex attribute with the shader `Colormap (Rainbow)` and `Colormap (Greyscale)`, but `RGB` does not work
### Steps to reproduce the bug
```python
1. Generate the tensorboard logs with the code below
2. Open tensorboard
3. As Data, select `modified_colors`
4. Use Shader `RGB`
#####################################################################################
import open3d as o3d
import numpy as np
# Monkey-patch torch.utils.tensorboard.SummaryWriter
from open3d.visualization.tensorboard_plugin import summary
# Utility function to convert Open3D geometry to a dictionary format
from open3d.visualization.tensorboard_plugin.util import to_dict_batch
from torch.utils.tensorboard import SummaryWriter
ply_point_cloud = o3d.data.PLYPointCloud()
pcd = o3d.io.read_point_cloud(ply_point_cloud.path)
logdir = "demo_logs/pytorch/small_scale"
writer = SummaryWriter(logdir)
pcd_dict = to_dict_batch([pcd])
# ADD ADDITIONAL VERTEX ATTRIBUTE
pcd_dict["vertex_modified_colors"] = np.copy(pcd_dict["vertex_colors"])
writer.add_3d('obj', pcd_dict, step=1)
writer.add_3d('obj', pcd_dict, step=2)
```
### Error message
Open3D WARNING] Tensor point cloud colors must have DType of Float32 not UInt8. Converting.
[Open3D INFO] OnDataChannelMessage: {"messageId":3,"class_name":"tensorboard/update_rendering","window_uid_list":["window_1"],"tag":"obj","render_state":{"property":"modified_colors","index":"0","shader":"defaultUnlit","colormap":{},"range":[0,255]},"updated":["shader"]}. Error processing message: TypeError: __eq__(): incompatible function arguments. The following argument types are supported:
1. (self: open3d.cpu.pybind.core.Dtype, arg0: open3d.cpu.pybind.core.Dtype) -> bool
Invoked with: UInt8,
At:
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/util.py(407): _normalize
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/util.py(670): apply
/home/manuel/anaconda3/lib/python3.11/site-packages/open3d/visualization/tensorboard_plugin/plugin.py(625): _update_rendering
### Expected behavior
The expectation is that a colored point cloud should be shown. Instead, it crashes
### Open3D, Python and System information
```markdown
- Operating system: Ubuntu 22.04
- Python version: 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0]
- Open3D version: 0.18.0 ( also tested on 0.18.0+553ca86)
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip
```
### Additional information
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.