isl-org / isl-org/Open3D

Possible mistake in how the extrinsic camera matrix is computed

Open
#6,522 0 comments 3 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### Describe the issue

I am trying to controll the camera in open3d by accessing the extrinsic camera parameters. While working with it I noticed a very peculiar behavior which I believe might be due to how the extrinsic camera parameters are presented. Instead of describing the transformation between word frame to camera frame oepn 3d provides the transformation between the camera frame to the word frame. Below I provide a minimal example where I just print the extrinsic transformation and its inverse, by playing around with the viewport you should be able to convince yourself of what I just said.

### Steps to reproduce the bug

```python
import open3d as o3d
import numpy as np

global_view = o3d.visualization.Visualizer()
global_view.create_window(window_name="Global view", visible=True)

coord_frame = o3d.geometry.TriangleMesh.create_coordinate_frame()
coord_frame.scale(100, center = coord_frame.get_center())
global_view.add_geometry(coord_frame)

while True:

print(o3d.__version__)
global_view.poll_events()
global_view.update_renderer()
camera_extrinsics = global_view.get_view_control().convert_to_pinhole_camera_parameters().extrinsic
print('raw camera extrinsic')
print(camera_extrinsics)
print('inverted extrinsic matrix')
print(np.linalg.inv(camera_extrinsics))
print('corrected extrinsic matrix')
print('-----------------------------------------------')
```

### Error message

_No response_

### Expected behavior

Open3d should provide the inverse of the extrinsic transformation when accessing it through `.get_view_control().convert_to_pinhole_camera_parameters().extrinsic` or the documentation should specify that the extrinsic is from the camera to the word frame

### Open3D, Python and System information

```markdown
- Operating system: Ubuntu 20.04
- Python version: Python 3.10.13
- Open3D version: output from python: 0.16.0, 0.17.0, 0.15.2, 0.14.1
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.