isl-org / isl-org/Open3D

High FPS point cloud visualization using Open3D

Open
#6,340 0 comments 2 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### My Question

My application requires rendering a stream of point clouds at high fps (>=30 fps) in C++. Each point cloud frame contains about 400K-800K points. Open3D feels like a good choice. Currently, I am hitting only 10-15 fps. Any suggestions are welcome.

## System:
Ubuntu 20.04 with NVIDIA GeForce RTX 3080 having cuda 11.3 installed. I installed open3D with flag BUILD_CUDA_MODULE=ON

## Issues:
- Only able to achieve 10-15 fps.
- Not sure if the visualizer is using GPU. How can I check if GPU is used?

## Code:

I have a thread that reads pointcloud >= 30fps and adds it to a queue `m_qOpen3D`. The visualizer thread pops color and position vectors and adds to Open3D point cloud object `pcd_ptr`. I set the camera viewpoints in the visualizer (every frame has a camera position) and render the point cloud frame.

```
typedef struct O3DData
{
vector points;
vector colors;
uint32_t frame_id;
}O3DData;
```

Visualization Thread:

```
visualization::Visualizer visualizer;
visualizer.CreateVisualizerWindow("Open3D", 1600, 900, 0, 0);
std::shared_ptr pcd_ptr = std::make_shared();

while(running)
{
O3DData *o3d_data = NULL;
while(running && !m_qOpen3D.pop(o3d_data))
sleep_ms(sleep_time);

cur_frame_id = o3d_data->frame_id;
UserData origData = m_dataPlayback->getUserData(cur_frame_id);

visualization::ViewControl& ctr = visualizer.GetViewControl();
ctr.ChangeFieldOfView(60);

camera::PinholeCameraParameters camera_params;
ctr.ConvertToPinholeCameraParameters(camera_params);

Eigen::Vector4d q_rotation{origData.quat[3], origData.quat[0], origData.quat[1], origData.quat[2]};
Eigen::Matrix3d R = geometry::Geometry3D::GetRotationMatrixFromQuaternion(q_rotation);
Eigen::Matrix4d temp;
temp << R.coeff(0, 0), R.coeff(0, 1), R.coeff(0, 2), origData.pos[0],
R.coeff(1, 0), R.coeff(1, 1), R.coeff(1, 2), origData.pos[1],
R.coeff(2, 0), R.coeff(2, 1), R.coeff(2, 2), origData.pos[2],
0, 0, 0, 1;
camera_params.extrinsic_ << temp.inverse();
ctr.ConvertFromPinholeCameraParameters(camera_params);

pcd_ptr->Clear();
pcd_ptr->points_ = o3d_data->points;
pcd_ptr->colors_ = o3d_data->colors;

if(first_frame)
{
visualizer.AddGeometry(pcd_ptr);
first_frame = false;
}
else
visualizer.UpdateGeometry(pcd_ptr);
visualizer.PollEvents();
visualizer.UpdateRender();

if(m_outputPath != "")
{
string output_file = FORMAT(m_outputPath << cur_frame_id << ".png");
visualizer.CaptureScreenImage(output_file);
LOG(INFO) << "Saved image to " << output_file;
}
delete o3d_data;
}
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.