isl-org / isl-org/Open3D

o3d.t.io.RSBagReader() with libRealSense Depth Post-Processing

Open
#6,164 3 comments 0 reactions 0 assignees View on GitHub
feature request good first issue
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### Proposed new feature or change

When reading a .bag file we would like apply [Depth Post-Processing](https://dev.intelrealsense.com/docs/depth-post-processing) as implemented in the Intel RealSense SDK 2.0 (libRealSense)

The existing [o3d.t.io.RSBagReader()](https://github.com/isl-org/Open3D/blob/master/cpp/open3d/t/io/sensor/realsense/RSBagReader.cpp) class reads a .bag file as-is but we cannot add the [libRealSense post processing](https://github.com/IntelRealSense/librealsense/blob/master/doc/post-processing-filters.md) such as decimation or temporal filters to the depth data.

When using o3d.t.io.RSBagReader() we get a RGBDImage Open3D format from [bag_reader.next_frame()](https://github.com/isl-org/Open3D/blob/a5be78cfe3da627ce6a38a7905e3e5cd6a64d811/cpp/pybind/t/io/sensor.cpp#L196). This means there is no space to actually apply depth post-processing.

This work involves adding additional code to o3d.t.io.RSBagReader() to apply additional filters to the depth variable. For example (Python Pseudo Code):

[depth_frame = frames.get_depth_frame();](https://github.com/isl-org/Open3D/blob/a5be78cfe3da627ce6a38a7905e3e5cd6a64d811/cpp/open3d/t/io/sensor/realsense/RSBagReader.cpp#L131)
decimation_filter = rs.decimation_filter()
decimation_filter.set_option(filter_magnitude, decimation_magnitude)
filtered_frame = decimation_filter.process(depth_frame)

Filters could be passed as json or a dictionary of filters

### References

https://github.com/oclis/ROS-EYE/blob/ec30974957be471ef60da47f173dfbbffcfc555d/src/realsense_device_manager.py#L49

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.