ByteDance-Seed / ByteDance-Seed/Depth-Anything-3
Questions about Unprojection in gs_adapter.py
- Dominant language
- Python
- Stars
- 6.3k
- Forks
- 702
- PR merge metrics
- No merged PRs in 30d
Description
## Questions about Unprojection in `gs_adapter.py`
I noticed that the GaussianAdapter module in gs_adapter.py calculates the positions of Gaussian points using:
`gs_means_world = origins + directions * gs_depths[..., None]`
However, the `directions` here are normalized in the `get_world_rays` function via:
`directions = directions / directions.norm(dim=-1, keepdim=True)`
This step has confused me because, based on the context, the depth value `gs_depths` should refer to **z-depth** rather than **distance-along-ray**. Additionally, in glb.py, unprojection is indeed performed using the **unnormalized ray direction** with the formula `K_inv @ pix * depth` (utils/export/glb.py Lines 239-240).
In my tests, I found that the 3DGS rendering effect remains normal when the code is left unchanged. Conversely, removing the ray direction normalization in `get_world_rays` causes distortion in the rendered images—even after removing the depth offset prediction in `gs_dpt`, which confirms that the distortion is not compensated for by the offset.
I would greatly appreciate your guidance and insights on this matter!
---
## 关于 gs_adapter.py 中反投影逻辑的疑问
我在查看 gs_adapter.py 的 GaussianAdapter 模块时发现,其计算高斯点世界坐标的公式为:
`gs_means_world = origins + directions * gs_depths[..., None]`
但其中的 `directions` 在 `get_world_rays` 函数中已做归一化处理:
`directions = directions / directions.norm(dim=-1, keepdim=True)`
这一处理让我产生疑惑:结合上下文来看,`gs_depths` 应为相机 z 轴深度(z-depth),而非射线长度(distance-along-ray);并且对比 `glb.py` 中的实现,其反投影确实采用**未归一化**的射线方向,公式为 `K_inv @ pix * depth`(对应 utils/export/glb.py 第 239–240 行)。
但我在实际测试中发现:
- 保留原有代码时,3DGS 渲染效果正常;
- 去掉 `get_world_rays` 中的射线方向归一化后,渲染图像反而出现畸变。
即便去掉 `gs_dpt` 中的深度偏移(depth offset)预测,畸变依然存在,说明该畸变并非由深度偏移补偿所致。
希望各位大佬不吝赐教!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing the ray construction and normalization in gs_adapter.py and get_world_rays with the unprojection formula in utils/export/glb.py at lines 239–240. Check how gs_depths and the gs_dpt depth offset are defined and used, then reproduce the reported rendering difference with and without normalization. Done means establishing whether the depth convention is consistent or documenting a confirmed discrepancy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100