isl-org / isl-org/Open3D

Full documentation for RGBD image datasets and possible API improvements.

Open
#4,888 0 comments 0 reactions 1 assignee Claimed by @reyanshsolis View on GitHub
feature request
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### Proposed new feature or change

Reading the camera info from `SampleRedwoodRGBDImages` and RGBD image datasets is non-trivial. The [current documentations](http://www.open3d.org/docs/latest/tutorial/data/index.html#rgbdimage) do not show how to load camera parameters.

Some problems to consider:
- How are the intrinsics loaded? What is the assumption?
- Shall we directly store the intrinsics in a text file instead of assuming "prime sense default"?
- How are the extrinsics stored? How are they stored in the original dataset? How shall we load this file in C++? If there is a convention used by the original author, probably we shall stick to that. Otherwise, how shall we store the extrinsics in the future? We have text format, json format, e.g. `PinholeCameraTrajectory::ConvertToJsonValue()`, and we can even define a new format.

Here's the example code to load the camera poses.

```python
import open3d as o3d
import numpy as np

class CameraPose:

def __init__(self, meta, mat):
self.metadata = meta
self.pose = mat

def __str__(self):
return 'Metadata : ' + ' '.join(map(str, self.metadata)) + '\n' + \
"Pose : " + "\n" + np.array_str(self.pose)

def read_trajectory(filename):
traj = []
with open(filename, 'r') as f:
metastr = f.readline()
while metastr:
metadata = list(map(int, metastr.split()))
mat = np.zeros(shape=(4, 4))
for i in range(4):
matstr = f.readline()
mat[i, :] = np.fromstring(matstr, dtype=float, sep=' \t')
traj.append(CameraPose(metadata, mat))
metastr = f.readline()
return traj

def main():
dataset = o3d.data.SampleRedwoodRGBDImages()
im_color = o3d.io.read_image(dataset.color_paths[0])
im_depth = o3d.io.read_image(dataset.depth_paths[0])
im_rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(
im_color, im_depth)

o3d_intrinsics = o3d.camera.PinholeCameraIntrinsic(
o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault)
camera_poses = read_trajectory(dataset.odometry_log_path)
pose = camera_poses[0].pose
extrinsics = np.linalg.inv(pose)
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
im_rgbd, o3d_intrinsics, extrinsics)
o3d.visualization.draw_geometries([pcd])

if __name__ == '__main__':
main()
```

### References

_No response_

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.