isl-org / isl-org/Open3D

CreateFromDepthImage uint16 confusion

Open
#6,962 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### Describe the issue

I get strange result from my realsense Z16 depth images, which has the depths stored as uint16_t (little-endian).
open3d PointCloud::CreateFromDepthImage only accepts num_channels == 1 and bytes_per_channel == 2 and converts the values then via Image::ConvertDepthToFloatImage. The other accepted format is float32 as bytes_per_channel_ == 4. This is fine.

But ConvertDepthToFloatImage does `float *p = output->PointerAt(x, y);` which accesses the data as float32 array for an uint16 image. Which looks wrong.
The final image looks good though. There must be some confusion somewhere, which I cannot find.

It should really convert the uint16 to float per value first.

### Steps to reproduce the bug

```c++
vector depths;
// ... read depths from a tiff
auto image = make_shared();
image->Prepare(width, height, /* num_channels */ 1, /* bytes_per_channel */ 2);
memcpy(image->data_.data(), depths.data(), depths.size() * 2);
open3d::camera::PinholeCameraIntrinsic intrinsics(
width, height, intr->f[0], intr->f[1], intr->pp[0], intr->pp[1]);
pcd = geometry::PointCloud::CreateFromDepthImage(*image, intrinsics);
```

### Error message

_No response_

### Expected behavior

_No response_

### Open3D, Python and System information

```markdown
- Operating system: Ubuntu 24.04
- Open3D version: latest main 553ca86cc11d57324b84eb96ae7cda5c90011091
- System architecture: x86
- Compiler version (if built from source): gcc 13
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.