rgbd_odometry_multi_scale gives inverse transformation
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
### Checklist
- [x] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [x] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [x] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).
### Describe the issue
According to the RGBD odometry documentation at https://github.com/isl-org/Open3D/blob/203ac47d99b7ead8a1b07535a67a29b71716cb51/cpp/open3d/t/pipelines/odometry/RGBDOdometry.h#L143-L144, `RGBDOdometryMultiScale` and hence `rgbd_odometry_multi_scale` estimate the transformation from **source** to **target**, or in other words, it gives the pose of the target data in the source frame.
But it seems to estimate the inverse of this.
### Steps to reproduce the bug
```python
#!/usr/bin/python3
import open3d as o3d
from open3d.t.pipelines.odometry import rgbd_odometry_multi_scale, Method
from open3d.t.geometry import Image, RGBDImage
import numpy as np
# fake RGBD images
i1 = np.zeros((120, 160, 3), dtype=np.uint8)
i2 = np.zeros((120, 160, 3), dtype=np.uint8)
d1 = (np.ones((120, 160)) * 1000).astype(np.uint16)
d2 = d1 + 10
d1[:10, :10] += 5
d2[:10, :10] += 5
rgbd1 = RGBDImage(
Image(np.ascontiguousarray(i1)),
Image(np.ascontiguousarray(d1)),
)
rgbd2 = RGBDImage(
Image(np.ascontiguousarray(i2)),
Image(np.ascontiguousarray(d2)),
)
intrinsics = np.array([
[80, 0, 80],
[ 0, 60, 60],
[ 0, 0, 1],
])
# optimized transformation matrix from source to target, inlier ratio, and fitness
# https://github.com/isl-org/Open3D/blob/203ac47d99b7ead8a1b07535a67a29b71716cb51/cpp/open3d/t/pipelines/odometry/RGBDOdometry.h#L143-L144
result = rgbd_odometry_multi_scale(
source = rgbd1,
target = rgbd2,
intrinsics = intrinsics,
init_source_to_target = o3d.core.Tensor(np.identity(4)),
depth_scale = 1000,
method = Method.PointToPlane,
)
print("transformation from source to target:")
print(result.transformation)
```
### Error message
```
transformation from source to target:
[[0.9999999999996052 -8.873573103987325e-07 4.515684927591465e-08 -0.008881941186502222],
[8.873573088095082e-07 0.9999999999996055 3.5326331110456846e-08 -0.008506522379392627],
[-4.515688062297498e-08 -3.5326291040182566e-08 0.9999999999999979 0.010000016679992096],
[0 0 0 1]]
Tensor[shape={4, 4}, stride={4, 1}, Float64, CPU:0, 0x160f0e00]
```
The estimated transformation sets the target (rgbd2) 1cm into the rgbd1 frame.
### Expected behavior
The fake data has the RGBD image 1 at 1 meter distance (1000 units) of a plane and RGBD image 2 at 1 meter and 1cm distance (1010 units). I.e. pointing at a plane, the camera would move 1cm backward from image 1 to image 2. Using the OpenCV camera frame convention (which seems to be used by Open3D too www.open3d.org/docs/0.7.0/python_api/open3d.geometry.create_point_cloud_from_depth_image.html), where z is pointing forward, this would mean a motion in negative z direction by 1cm.
With `source = rgbd1` and `target = rgbd2`, the transformation from source to target should be -1cm in z direction, but the estimated transformation is the opposite.
### Open3D, Python and System information
```markdown
- Operating system: Ubuntu 24.04
- Python version: Python 3.12
- Open3D version: output from python: 0.19.0
- System architecture: x86
- How did you install Open3D?: pip
```
### Additional information
The `ComputeOdometryResultPointToPlane` mentions that the delta transformation is "target_to_source":
https://github.com/isl-org/Open3D/blob/203ac47d99b7ead8a1b07535a67a29b71716cb51/cpp/open3d/t/pipelines/odometry/RGBDOdometry.cpp#L400
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.