Full documentation for RGBD image datasets and possible API improvements.
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
### Checklist
- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).
### Proposed new feature or change
Reading the camera info from `SampleRedwoodRGBDImages` and RGBD image datasets is non-trivial. The [current documentations](http://www.open3d.org/docs/latest/tutorial/data/index.html#rgbdimage) do not show how to load camera parameters.
Some problems to consider:
- How are the intrinsics loaded? What is the assumption?
- Shall we directly store the intrinsics in a text file instead of assuming "prime sense default"?
- How are the extrinsics stored? How are they stored in the original dataset? How shall we load this file in C++? If there is a convention used by the original author, probably we shall stick to that. Otherwise, how shall we store the extrinsics in the future? We have text format, json format, e.g. `PinholeCameraTrajectory::ConvertToJsonValue()`, and we can even define a new format.
Here's the example code to load the camera poses.
```python
import open3d as o3d
import numpy as np
class CameraPose:
def __init__(self, meta, mat):
self.metadata = meta
self.pose = mat
def __str__(self):
return 'Metadata : ' + ' '.join(map(str, self.metadata)) + '\n' + \
"Pose : " + "\n" + np.array_str(self.pose)
def read_trajectory(filename):
traj = []
with open(filename, 'r') as f:
metastr = f.readline()
while metastr:
metadata = list(map(int, metastr.split()))
mat = np.zeros(shape=(4, 4))
for i in range(4):
matstr = f.readline()
mat[i, :] = np.fromstring(matstr, dtype=float, sep=' \t')
traj.append(CameraPose(metadata, mat))
metastr = f.readline()
return traj
def main():
dataset = o3d.data.SampleRedwoodRGBDImages()
im_color = o3d.io.read_image(dataset.color_paths[0])
im_depth = o3d.io.read_image(dataset.depth_paths[0])
im_rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(
im_color, im_depth)
o3d_intrinsics = o3d.camera.PinholeCameraIntrinsic(
o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault)
camera_poses = read_trajectory(dataset.odometry_log_path)
pose = camera_poses[0].pose
extrinsics = np.linalg.inv(pose)
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
im_rgbd, o3d_intrinsics, extrinsics)
o3d.visualization.draw_geometries([pcd])
if __name__ == '__main__':
main()
```
### References
_No response_
### Additional information
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.