ByteDance-Seed / ByteDance-Seed/Depth-Anything-3

Questions about Unprojection in gs_adapter.py

Open
#236 1 comment 4 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.3k
Forks
702
PR merge metrics
No merged PRs in 30d

Description

## Questions about Unprojection in `gs_adapter.py`

I noticed that the GaussianAdapter module in gs_adapter.py calculates the positions of Gaussian points using:
`gs_means_world = origins + directions * gs_depths[..., None]`

However, the `directions` here are normalized in the `get_world_rays` function via:
`directions = directions / directions.norm(dim=-1, keepdim=True)`

This step has confused me because, based on the context, the depth value `gs_depths` should refer to **z-depth** rather than **distance-along-ray**. Additionally, in glb.py, unprojection is indeed performed using the **unnormalized ray direction** with the formula `K_inv @ pix * depth` (utils/export/glb.py Lines 239-240).

In my tests, I found that the 3DGS rendering effect remains normal when the code is left unchanged. Conversely, removing the ray direction normalization in `get_world_rays` causes distortion in the rendered images—even after removing the depth offset prediction in `gs_dpt`, which confirms that the distortion is not compensated for by the offset.

I would greatly appreciate your guidance and insights on this matter!

---
## 关于 gs_adapter.py 中反投影逻辑的疑问

我在查看 gs_adapter.py 的 GaussianAdapter 模块时发现,其计算高斯点世界坐标的公式为:
`gs_means_world = origins + directions * gs_depths[..., None]`

但其中的 `directions` 在 `get_world_rays` 函数中已做归一化处理:
`directions = directions / directions.norm(dim=-1, keepdim=True)`

这一处理让我产生疑惑:结合上下文来看,`gs_depths` 应为相机 z 轴深度(z-depth),而非射线长度(distance-along-ray);并且对比 `glb.py` 中的实现,其反投影确实采用**未归一化**的射线方向,公式为 `K_inv @ pix * depth`(对应 utils/export/glb.py 第 239–240 行)。

但我在实际测试中发现:
- 保留原有代码时,3DGS 渲染效果正常;
- 去掉 `get_world_rays` 中的射线方向归一化后,渲染图像反而出现畸变。

即便去掉 `gs_dpt` 中的深度偏移(depth offset)预测,畸变依然存在,说明该畸变并非由深度偏移补偿所致。

希望各位大佬不吝赐教!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the ray construction and normalization in gs_adapter.py and get_world_rays with the unprojection formula in utils/export/glb.py at lines 239–240. Check how gs_depths and the gs_dpt depth offset are defined and used, then reproduce the reported rendering difference with and without normalization. Done means establishing whether the depth convention is consistent or documenting a confirmed discrepancy.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.