o3d.t.io.RSBagReader() with libRealSense Depth Post-Processing
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
### Checklist
- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).
### Proposed new feature or change
When reading a .bag file we would like apply [Depth Post-Processing](https://dev.intelrealsense.com/docs/depth-post-processing) as implemented in the Intel RealSense SDK 2.0 (libRealSense)
The existing [o3d.t.io.RSBagReader()](https://github.com/isl-org/Open3D/blob/master/cpp/open3d/t/io/sensor/realsense/RSBagReader.cpp) class reads a .bag file as-is but we cannot add the [libRealSense post processing](https://github.com/IntelRealSense/librealsense/blob/master/doc/post-processing-filters.md) such as decimation or temporal filters to the depth data.
When using o3d.t.io.RSBagReader() we get a RGBDImage Open3D format from [bag_reader.next_frame()](https://github.com/isl-org/Open3D/blob/a5be78cfe3da627ce6a38a7905e3e5cd6a64d811/cpp/pybind/t/io/sensor.cpp#L196). This means there is no space to actually apply depth post-processing.
This work involves adding additional code to o3d.t.io.RSBagReader() to apply additional filters to the depth variable. For example (Python Pseudo Code):
[depth_frame = frames.get_depth_frame();](https://github.com/isl-org/Open3D/blob/a5be78cfe3da627ce6a38a7905e3e5cd6a64d811/cpp/open3d/t/io/sensor/realsense/RSBagReader.cpp#L131)
decimation_filter = rs.decimation_filter()
decimation_filter.set_option(filter_magnitude, decimation_magnitude)
filtered_frame = decimation_filter.process(depth_frame)
Filters could be passed as json or a dictionary of filters
### References
https://github.com/oclis/ROS-EYE/blob/ec30974957be471ef60da47f173dfbbffcfc555d/src/realsense_device_manager.py#L49
### Additional information
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.