isl-org / isl-org/Open3D

How to convert `open3d.cuda.pybind.geometry.trianglemesh` `open3d.visualization.rendering.TriangleMeshModel`?

Open
#6,388 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

Hi, I am trying to visualize a camera pose in 3D pointcloud in Open3D by using `open3d.visualization.O3DVisualizer` and `import open3d.visualization.gui` . A camera pose is created by using this function,

```
def draw_camera(K, R, t, w, h,
scale=1, color=[0.8, 0.2, 0.8]):
"""Create axis, plane and pyramed geometries in Open3D format.
:param K: calibration matrix (camera intrinsics)
:param R: rotation matrix
:param t: translation
:param w: image width
:param h: image height
:param scale: camera model scale
:param color: color of the image plane and pyramid lines
:return: camera model geometries (axis, plane and pyramid)
"""

# intrinsics
K = K.copy() / scale
Kinv = np.linalg.inv(K)

# 4x4 transformation
T = np.column_stack((R, t))
T = np.vstack((T, (0, 0, 0, 1)))

# axis
axis = open3d.geometry.TriangleMesh.create_coordinate_frame(
size=0.5 * scale)
axis.transform(T)

# points in pixel
points_pixel = [
[0, 0, 0],
[0, 0, 1],
[w, 0, 1],
[0, h, 1],
[w, h, 1],
]

# pixel to camera coordinate system
points = [Kinv @ p for p in points_pixel]

# image plane
width = abs(points[1][0]) + abs(points[3][0])
height = abs(points[1][1]) + abs(points[3][1])
plane = open3d.geometry.TriangleMesh.create_box(width, height, depth=1e-6)
plane.paint_uniform_color(color)
plane.translate([points[1][0], points[1][1], scale])
plane.transform(T)

# pyramid
points_in_world = [(R @ p + t) for p in points]
lines = [
[0, 1],
[0, 2],
[0, 3],
[0, 4],
]
colors = [color for i in range(len(lines))]
line_set = open3d.geometry.LineSet(
points=open3d.utility.Vector3dVector(points_in_world),
lines=open3d.utility.Vector2iVector(lines))
line_set.colors = open3d.utility.Vector3dVector(colors)

# return as list in Open3D format
return [axis, plane, line_set]
```

I am trying to visualize the trianglemesh from this function in
```
app = gui.Application.instance
app.initialize()

points = self.create_point_cloud()

vis = open3d.visualization.O3DVisualizer("Environment PointCloud", 1024, 768)
vis.show_settings = True
vis.add_geometry("Points", points)

# camera Poses
** create frames

# camera_poses is a list of open3d.cuda.pybind.geometry.trianglemesh objects.
for fidx, camera_pose in enumerate(camera_poses):
vis.add_geometry(name="camera{}".format(fidx),
model=camera_pose,
group="Camera Poses",
time=0.0,
is_visible=True)
```
But I am getting this error,
```
TypeError: add_geometry(): incompatible function arguments. The following argument types are supported:
1. (self: open3d.cuda.pybind.visualization.O3DVisualizer, name: str, geometry: open3d.cuda.pybind.geometry.Geometry3D, material: open3d.cuda.pybind.visualization.rendering.MaterialRecord = None, group: str = '', time: float = 0.0, is_visible: bool = True) -> None
2. (self: open3d.cuda.pybind.visualization.O3DVisualizer, name: str, geometry: open3d.cuda.pybind.t.geometry.Geometry, material: open3d.cuda.pybind.visualization.rendering.MaterialRecord = None, group: str = '', time: float = 0.0, is_visible: bool = True) -> None
3. (self: open3d.cuda.pybind.visualization.O3DVisualizer, name: str, model: open3d.cuda.pybind.visualization.rendering.TriangleMeshModel, material: open3d.cuda.pybind.visualization.rendering.MaterialRecord = None, group: str = '', time: float = 0.0, is_visible: bool = True) -> None
4. (self: object, arg0: dict) -> None

Invoked with: ; kwargs: name='frame0', model=TriangleMesh with 1134 points and 2240 triangles., group='frames', time=0.0, is_visible=True
```

Previously, I was using `open3d.visualization.Visualizer` api to visualize this trianglemesh model by using `add_geometry` method and it is okay but if I switch `open3d.visualization.O3DVisualizer`, this error happens.

The camera pose trianglemesh that I want to visualize is something like this:
![pose](https://github.com/isl-org/Open3D/assets/42746452/590866df-4f5e-4025-ac40-2e89c9f5a90d)

Thank you for the help in advance!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.