Error in Reconstruction System using Intel RealSense Depth Camera D435i
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
**Describe the bug**
I had some issues when trying the reconstruction system using Intel RealSense D435i camera.
After I try both of the following [Input arguments](http://www.open3d.org/docs/release/tutorial/reconstruction_system/capture_your_own_dataset.html#input-arguments) below (one by one):
`python realsense_recorder.py --record_imgs`
`python realsense_recorder.py --record_rosbag`
the `color` and `depth` folder can be produced by `python realsense_recorder.py --record_imgs`
and also `realsense.bag` can be produced by `python realsense_recorder.py --record_rosbag`
But when I try to:
- #1. _Run reconstruction system from RGBD dataset_ (color and depth folder) like in the [Quick start](http://www.open3d.org/docs/release/tutorial/reconstruction_system/system_overview.html#quick-start)
`python run_system.py config_file --make --register --refine --integrate`
where my `realsense.json` configuration is:
```
{
"name": "Realsense bag file",
"path_dataset": "dataset/realsense",
"path_intrinsic": "",
"max_depth": 3.0,
"voxel_size": 0.05,
"max_depth_diff": 0.07,
"preference_loop_closure_odometry": 0.1,
"preference_loop_closure_registration": 5.0,
"tsdf_cubic_size": 3.0,
"icp_method": "color",
"global_registration": "ransac",
"python_multi_threading": true
}
```
It runs pretty well and producing like the [Results](http://www.open3d.org/docs/release/tutorial/reconstruction_system/integrate_scene.html#results) with my own dataset
_But it was blur and had distortion even when I do record with slow, steady, and longer time._
And sometimes it creates **scattered point cloud** (shown in the screenshots with blue and red mark as stuff that I have scanned).
- #2. _[Run reconstruction system](http://www.open3d.org/docs/release/tutorial/reconstruction_system/capture_your_own_dataset.html#run-reconstruction-system) from the `realsense.bag`_ with `realsense.json` configuration below:
```
{
"name": "Realsense bag file",
"path_dataset": "dataset/realsense/realsense.bag",
"path_intrinsic": "",
"max_depth": 3.0,
"voxel_size": 0.05,
"max_depth_diff": 0.07,
"preference_loop_closure_odometry": 0.1,
"preference_loop_closure_registration": 5.0,
"tsdf_cubic_size": 3.0,
"icp_method": "color",
"global_registration": "ransac",
"python_multi_threading": true
}
```
the script `python run_system.py config/realsense.json --make --register --refine --integrate` was resulting error like this:
```
Extracting frames from RGBD video file
../utility\file.py:126: UserWarning: Skipping frame extraction for dataset/realsense/realsense.bag since files are present.
warn(f"Skipping frame extraction for {rgbd_video_file} since files are"
====================================
Configuration
====================================
name : Realsense bag file
path_dataset : dataset/realsense\realsense
path_intrinsic : dataset/realsense\realsense\intrinsic.json
max_depth : 3.0
voxel_size : 0.05
max_depth_diff : 0.07
preference_loop_closure_odometry : 0.1
preference_loop_closure_registration : 5.0
tsdf_cubic_size : 3.0
icp_method : color
global_registration : ransac
python_multi_threading : True
depth_map_type : redwood
n_frames_per_fragment : 100
n_keyframes_per_n_frame : 5
min_depth : 0.3
depth_scale : 999.9999389648438
max_iterations : 5
sdf_trunc : 0.04
block_count : 40000
distance_threshold : 0.07
fitness_threshold : 0.3
regularizer_weight : 1
method : slac
device : cpu:0
save_output_as : pointcloud
folder_slac : slac/
template_optimized_posegraph_slac : optimized_posegraph_slac.json
folder_fragment : fragments/
subfolder_slac : slac/0.050/
template_fragment_posegraph : fragments/fragment_%03d.json
template_fragment_posegraph_optimized : fragments/fragment_optimized_%03d.json
template_fragment_pointcloud : fragments/fragment_%03d.ply
folder_scene : scene/
template_global_posegraph : scene/global_registration.json
template_global_posegraph_optimized : scene/global_registration_optimized.json
template_refined_posegraph : scene/refined_registration.json
template_refined_posegraph_optimized : scene/refined_registration_optimized.json
template_global_mesh : scene/integrated.ply
template_global_traj : scene/trajectory.log
debug_mode : False
OpenCV is detected. Using ORB + 5pt algorithm
making fragments from RGBD sequence.
Traceback (most recent call last):
File "run_system.py", line 91, in
make_fragments.run(config)
File "D:\E-Learning\Visual Studio 2019\source\repos\Open3D\examples\python\reconstruction_system\make_fragments.py", line
185, in run
Parallel(n_jobs=MAX_THREAD)(delayed(process_single_fragment)(
File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\parallel.py", line 966, in __call__
n_jobs = self._initialize_backend()
File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\parallel.py", line 733, in _initialize_backend
n_jobs = self._backend.configure(n_jobs=self.n_jobs, parallel=self,
File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\_parallel_backends.py", line 489, in configure
n_jobs = self.effective_n_jobs(n_jobs)
File "C:\ProgramData\Anaconda3\lib\site-packages\joblib\_parallel_backends.py", line 504, in effective_n_jobs
raise ValueError('n_jobs == 0 in Parallel has no meaning')
ValueError: n_jobs == 0 in Parallel has no meaning
```
**To Reproduce**
Steps to reproduce the behavior:
- for issue #1:
1. cd 'the path to `reconstruction_system/sensors`'
2. python `python realsense_recorder.py --record_imgs`
3. capturing, after that, close with press `esc` (I do scanning until ±6000 color + depth image captured)
4. realsense folder created
```
realsense\
color\rgb_files.jpg
depth\depth_files.png
camera_intrinsic.json
```
where `camera_instrinsic.json` by default is:
```
{
"width": 640,
"height": 480,
"intrinsic_matrix": [
613.3899536132812,
0,
0,
0,
613.7398071289062,
0,
328.3892822265625,
245.40103149414062,
1
]
}
```
5. cd `..`
6. python `python run_system.py config/realsense.json --make --register --refine --integrate`
with `realsense.json` configuration:
```
{
"name": "Realsense bag file",
"path_dataset": "dataset/realsense",
"path_intrinsic": "",
"max_depth": 3.0,
"voxel_size": 0.05,
"max_depth_diff": 0.07,
"preference_loop_closure_odometry": 0.1,
"preference_loop_closure_registration": 5.0,
"tsdf_cubic_size": 3.0,
"icp_method": "color",
"global_registration": "ransac",
"python_multi_threading": true
}
```
7. fragments and scene folder created
8. I create `visualize_point_cloud.ipynb` which inside were extracted from [Visualize point cloud](http://www.open3d.org/docs/release/tutorial/geometry/pointcloud.html#Visualize-point-cloud) and change the `read_point_cloud` path to `../reconstruction_system/dataset/realsense/scene/integrated.ply`
9. The result was in the **scattered point cloud** screenshots with blue and red marks
- for issue #2:
1. cd 'the path to `reconstruction_system/dataset`'
2. `mkdir realsense`
3. cd `../sensors`
4. python `python realsense_recorder.py --record_rosbag`
5. capturing, after that, close with press `esc`
6. `realsense.bag` created in `reconstruction_system/dataset/realsense`
7. configure `path_dataset` in `realsense.json` from `"dataset/realsense"` to `"dataset/realsense/realsense.bag"`
8. cd `..`
9. python `python run_system.py config/realsense.json --make --register --refine --integrate`
10. resulting an error as mentioned above (`ValueError: n_jobs == 0 in Parallel has no meaning`), with new `realsense` directory like below:
```
realsense\
realsense\
color\
depth\
fragments\
intrinsic.json
realsense.bag
```
where `intrinsic.json` by default is:
```
{
"color_format" : "BGR8",
"depth_format" : "Z16",
"depth_scale" : 999.99993896484375,
"device_name" : "Intel RealSense D435I",
"fps" : 30.0,
"height" : 480,
"intrinsic_matrix" :
[
613.38995361328125,
0.0,
0.0,
0.0,
613.73980712890625,
0.0,
328.3892822265625,
245.40103149414063,
1.0
],
"serial_number" : "918512071170",
"stream_length_usec" : 4946741,
"width" : 640
}
```
**Expected behavior**
- for issue #1: pointcloud constructed without scattered and the result isn't blur or distorted
- for issue #2: does it should be like the documentation [Running the scence reconstruction pipeline](http://www.open3d.org/docs/release/tutorial/sensor/realsense.html#running-the-scene-reconstruction-pipeline) (with my own .bag file)? if yes, then how to do that using my own `.bag` file?
**Screenshots**


**Environment:**
- Operating system: Windows 10 64-bit
- Python version: Python 3.8
- Open3D version: 0.12.0
- Is this remote workstation?: no
- How did you install Open3D?: pip
**Additional context**
- Do you have suggestion to repair the scattered pointcloud and error (from .bag file) above?
- Any configuration needed that should I set for the camera?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.