ByteDance-Seed / ByteDance-Seed/Depth-Anything-3
Discrepancy in Ray Map channel ordering between Paper and Code implementation
- Dominant language
- Python
- Stars
- 6.3k
- Forks
- 702
- PR merge metrics
- No merged PRs in 30d
Description
I have noticed a discrepancy regarding the definition of the 6-channel Ray Map (`camray`) between the paper description and the actual code implementation.
**1. Paper Definition**
In the paper, the definition of the camera ray is stated as follows:
> "For each pixel p, the camera ray r ∈ R6 is defined by its origin t ∈ R3 and direction d ∈ R3: r = (t, d)"
This implies that the channel ordering is **[Origin, Direction]** (i.e., first 3 channels are $t$, last 3 channels are $d$).
**2. Code Implementation**
However, looking at `ray_utils.py` and `da3.py`, the implementation follows the order **[Direction, Origin]**.
**Evidence from `ray_utils.py`:**
In the `camray_to_caminfo()` function, the first 3 channels are used to solve for the rotation matrix (implying direction), while the last 3 channels are used to solve for translation (implying origin).
```python
# camray shape: (B, S, num_patches_y, num_patches_x, 6)
# Channels 0-3: Ray Direction
# Used for solving Rotation (R) and Intrinsics via RANSAC
ray_direction = camray[:, :, :3]
# Channels 3-6: Ray Origin
# Used for calculating Translation (T)
# The code calculates T by weighted averaging of the last 3 channels
T = torch.sum(camray[:, :, 3:] * confidence.unsqueeze(-1), dim=1) / torch.sum(confidence, ...)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.