isl-org / isl-org/Open3D

Panorama 360 RGBD to point cloud

Open
#7,047 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [ ] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [ ] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### My Question

Hi,

I have a 360° panorama image and a depth map, but when I try to extract the point cloud and display it, the output doesn’t appear correctly. Is it possible to fix this or provide some information?

I need the display to look like a typical 360° panorama viewer, not the current incorrect output.

Thanks!

I'm using C++ and here is my code:
```
std::shared_ptr my_depthTo3DPointCloud_panorama(
const cv::Mat& depthMap, const cv::Mat& rawImage, const cv::Mat& mask = cv::Mat(), float depth_scale = 1.0f) {

auto point_cloud = std::make_shared();

// Get dimensions of the depth map
int height = depthMap.rows;
int width = depthMap.cols;

// Panorama field of view (adjust as necessary for your specific image)
const float fov_x = 2.0f * M_PI; // 360 degrees horizontally
const float fov_y = M_PI; // 180 degrees vertically

for (int v = 0; v < height; ++v) {
for (int u = 0; u < width; ++u) {
if (mask.empty() || mask.at(v, u) > 0) {
// Retrieve depth value (assuming uchar depth map, scale it for precision)
float depth = static_cast(depthMap.at(v, u)) * depth_scale;

// Calculate angles for spherical mapping
float theta = (static_cast(u) / width - 0.5f) * fov_x; // Azimuth angle
float phi = (0.5f - static_cast(v) / height) * fov_y; // Elevation angle

// Convert spherical coordinates to Cartesian coordinates
float x = depth * cos(phi) * sin(theta);
float y = depth * sin(phi);
float z = depth * cos(phi) * cos(theta);

// Add the 3D point
point_cloud->points_.emplace_back(x, y, z);

// Extract color information from the original image
cv::Vec3b color = rawImage.at(v, u);
point_cloud->colors_.emplace_back(color[2] / 255.0, color[1] / 255.0, color[0] / 255.0);
}
}
}

return point_cloud;
}
```

![image](https://github.com/user-attachments/assets/a703e303-f1e8-403c-8262-eebefc4a31bc)

point cloud
![image](https://github.com/user-attachments/assets/5e0ba065-ddf5-4fa3-9820-8c8d55895197)

![image](https://github.com/user-attachments/assets/312f0253-f017-44ae-a861-a2e86551cd68)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.