isl-org / isl-org/Open3D

Compatibility between Open3D camera and gsplat/Gaussian Splatting

Open
#7,141 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [x] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [x] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [x] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### My Question

Hello, I'm trying to combine Open3D with an image rendered from gsplat (Gaussian Splatting).

Gaussian Splatting coordinate system is (+X, -Y, +Z), the intrinsic matrix is calculated as follows (row-major):

```py
K = np.zeros((3, 3), dtype=np.float64)
K[0, 0] = fx * W * 0.5
K[1, 1] = fy * H * 0.5
K[2, 0] = W * 0.5
K[2, 1] = H * 0.5
K[2, 2] = 1
```

The extrinsic is:

```py
inv(Rot * t)
```

Rendering the Gaussian Splatting **pointcloud** with Open3D, using the same extrinsic_matrix/intrinsic_matrix that I use in gsplat, gives the following result:

I'm rendering using:

```py
o3d.visualization.draw(
[pointcloud],
width=W,
height=H,
field_of_view=None,
intrinsic_matrix=intrinsic,
extrinsic_matrix=extrinsic_params,
)
```

Is there anything, like a zoom, or any parameter transforming the camera matrices that I'm missing? 👀

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.