isl-org / isl-org/Open3D

`compute_unique_block_coordinates()` seems inconsistent with CUDA VoxelBlockGrid

Open
#7,497 3 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [x] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [x] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [x] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### Describe the issue

Hi,

I might have run into an issue with `VoxelBlockGrid.compute_unique_block_coordinates()` when using a CUDA VoxelBlockGrid from Python.

Setup:

* Open3D built from source with CUDA enabled
* Python API
* `VoxelBlockGrid` created on `CUDA:0`

Creating the VBG itself works fine:

```python
vbg = o3d.t.geometry.VoxelBlockGrid(
attr_names=("occ", "free", "weight"),
attr_dtypes=(
o3d.core.float32,
o3d.core.float32,
o3d.core.float32,
),
attr_channels=((1,), (1,), (1,)),
voxel_size=0.01,
block_resolution=16,
block_count=200000,
device=o3d.core.Device("CUDA:0"),
)
```

The issue appears when calling:

```python
vbg.compute_unique_block_coordinates(...)
```

If I pass CPU tensors/images, I get:

```text
ParallelFor for CUDA cannot run on device CPU:0
```

which suggests the CUDA kernel is selected because the VBG itself is on CUDA.

But if I move the inputs to CUDA, I instead get:

```text
Tensor has device CUDA:0, but is expected to have CPU:0
```

coming from `InverseTransformation()` internally.

So it looks like:

* part of the implementation expects CPU tensors
* but kernel dispatch depends on the VBG device

which makes the CUDA VBG case difficult to use from Python.

Maybe I am misunderstanding the expected usage, but I thought I should report it in case this is an actual CUDA path inconsistency.

Thanks.

### Steps to reproduce the bug

```python
import numpy as np
import open3d as o3d

print(o3d.core.cuda.is_available())

device = o3d.core.Device("CUDA:0")

#
# CUDA VoxelBlockGrid
#

vbg = o3d.t.geometry.VoxelBlockGrid(
attr_names=("tsdf", "weight"),
attr_dtypes=(
o3d.core.float32,
o3d.core.float32,
),
attr_channels=((1,), (1,)),
voxel_size=0.01,
block_resolution=16,
block_count=1000,
device=device,
)

#
# dummy depth image
#

depth_np = np.ones(
(32, 32),
dtype=np.float32,
)

#
# CASE 1:
# CPU tensors
#

print("\nCASE 1: CPU tensors\n")

depth_cpu = o3d.t.geometry.Image(
o3d.core.Tensor(
depth_np,
device=o3d.core.Device("CPU:0"),
)
)

intrinsic_cpu = o3d.core.Tensor(
np.eye(3, dtype=np.float64),
device=o3d.core.Device("CPU:0"),
)

extrinsic_cpu = o3d.core.Tensor(
np.eye(4, dtype=np.float64),
device=o3d.core.Device("CPU:0"),
)

try:

coords = vbg.compute_unique_block_coordinates(
depth_cpu,
intrinsic_cpu,
extrinsic_cpu,
1000.0,
5.0,
8.0,
)

print("CPU tensors worked")

except Exception as e:

print("CPU tensors failed:")
print(e)

#
# CASE 2:
# CUDA tensors
#

print("\nCASE 2: CUDA tensors\n")

depth_cuda = o3d.t.geometry.Image(
o3d.core.Tensor(
depth_np,
device=device,
)
)

intrinsic_cuda = o3d.core.Tensor(
np.eye(3, dtype=np.float64),
device=device,
)

extrinsic_cuda = o3d.core.Tensor(
np.eye(4, dtype=np.float64),
device=device,
)

try:

coords = vbg.compute_unique_block_coordinates(
depth_cuda,
intrinsic_cuda,
extrinsic_cuda,
1000.0,
5.0,
8.0,
)

print("CUDA tensors worked")

except Exception as e:

print("CUDA tensors failed:")
print(e)
```

### Error message

CASE 1: CPU tensors

```
CPU tensors failed:
[Open3D Error] (void open3d::core::ParallelForCUDA_(const open3d::core::Device&, int64_t, const func_t&) [with func_t = __nv_dl_wrapper_t<__nv_dl_tag&, const open3d::core::Tensor&, const open3d::core::Tensor&, const open3d::core::Tensor&, open3d::core::Tensor&, int, float, float, float, float, int), open3d::t::geometry::kernel::voxel_grid::DepthTouchCUDA, 1>, int, int, open3d::t::geometry::kernel::TArrayIndexer, float, float, open3d::t::geometry::kernel::TransformIndexer, float, const int, int*, float, int*>; int64_t = long int]) /home/pierre/workspace/Open3D/cpp/open3d/core/ParallelFor.h:52: ParallelFor for CUDA cannot run on device CPU:0.
```

CASE 2: CUDA tensors

```
CUDA tensors failed:
[Open3D Error] (open3d::core::Tensor open3d::t::geometry::InverseTransformation(const open3d::core::Tensor&)) /home/pierre/workspace/Open3D/cpp/open3d/t/geometry/Utility.h:80: Tensor has device CUDA:0, but is expected to have CPU:0.
```

### Expected behavior

_No response_

### Open3D, Python and System information

```markdown
- Operating system: Ubuntu 22.04
- Python version: Python 3.10.12
- Open3D version: output from python: 0.19.0+241aaeea0
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?:build from source
- Compiler version (if built from source): gcc 11
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.