isl-org / isl-org/Open3D

How to convert Open3d camera Pose to unity

Open
#4,641 15 comments 0 reactions 0 assignees View on GitHub
geometry question transform
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

Greetings!
I've been running thru an issue where I do a camera extrinsic calibration in Open3D with global registration and ICP, the point clouds from each camera get aligned correctly but when I pass the camera pose to unity I get a really bad result. The Open3D Transform seems fine I already inverted the Y axis due to the difference in the coordinate system but I can't figure out where the variation error is coming from, is it just because of imprecision, or I'm missing something.

I invert the transform so I can extract the rotation matrix and the translation vector from the pose. My approach is to get the rotation and the translation and assign them to their correspondent kinect object in unity. The main Kinect stays in the origin of the coordinates and then I apply the transformations to each camera referencing the main one. I'll attach some data and captures of the setup.

![image](https://user-images.githubusercontent.com/95495428/150535084-a67d634c-c684-42e0-8291-cc67d66a0055.png)

right_transformation
array([[ 0.96518757, -0.24386752, -0.09455995, -0.30049289],
[ 0.14996032, 0.21974007, 0.9639638 , -0.97258158],
[-0.21430085, -0.94458612, 0.24866084, 0.60323834],
[ 0. , 0. , 0. , 1. ]])
-----------------------------------------------------------------------
First set of vectors:
**rotation angles right: [ 75.53550248 5.42599294 -14.17977523]
translation right: [[0.56515514 -0.71024525 0.75911709]]**

left_transformation
array([[ 0.9690198 , 0.23938183, -0.06080263, -0.36967974],
[-0.15101459, 0.37945827, -0.91280119, 1.0112917 ],
[-0.19543595, 0.89370452, 0.40385273, 0.61812839],
[ 0. , 0. , 0. , 1. ]])
--------------------------------------------------------------------
First set of vectors:
**rotation angles left: [-66.13385402 3.48588443 13.87624411]
translation left: [[ 0.63175131 0.84767252 0.65099793]]**

The translation in unity kinda makes sense with the actual setup, but can't get a correct result.

![setup](https://user-images.githubusercontent.com/95495428/150542619-e63343d5-02a2-4cd4-a278-8ea99bcb2f92.jpg)

When I manually calibrate the cameras in unity I get these vectors, and I really can't figure out if there's any correlation with the ones that I mentioned previously.

Second set of vectors:
**rotation angles right: [ 72.861, 7.016, 20.971]
translation right: [0.3640004, 0.9750017, 0.832]**
**rotation angles left: [-76.012, 22.273, -35.441]
translation left: [ 0.331, -1.025, 0.556]**

![image](https://user-images.githubusercontent.com/95495428/150541953-d363f2b1-7c86-40eb-8d78-cccabdbe2946.png)

Any help or tip about what's happening would be greatly appreciated since I've been stuck with this for a long time,
Thanks in advance!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.