open3d point cloud rendering doesn't match opencv 2D projection
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
### Checklist
- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).
### My Question
I have a point cloud already transformed into the camera coordinate frame so the projection matrix should be pretty much the same as the intrinsics . My attempts to render the point cloud does not match my projection written in OpenCV (as the point cloud is already aligned to the camera coordinate frame, I can project it into 2D directly using values from the camera instrinsic matrix in OpenCV).
My code for rendering the point cloud `pcl` via Open3D is as follows:
```
img_width = 640
img_height = 512
render = o3d.visualization.rendering.OffscreenRenderer(width=img_width, height=img_height)
render.scene.set_background([0.0, 0.0, 0.0, 1.0]) # RGBA
mtl = o3d.visualization.rendering.MaterialRecord()
mtl.base_color = [1.0, 1.0, 1.0, 1.0] # RGBA
mtl.shader = "defaultUnlit"
render.scene.add_geometry("model", pcl, mtl)
extrinsics_matrix = np.eye(4)
render.setup_camera(intrinsics_matrix, extrinsics_matrix, img_width, img_height)
center = [0, 0, 1]
eye = [0, 0, 0]
up = [0, 1, 0]
render.scene.camera.look_at(center, eye, up)
output = render.render_to_image()
```
Open3D rendering:
vs OpenCV projection:
There seems to be translation and possibly scale differences between the two outputs.
Lastly, I'm confused what the difference is between `render.setup_camera`, `render.scene.camera.set_projection`, and `render.scene.camera.look_at`.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.