ByteDance-Seed / ByteDance-Seed/Depth-Anything-3

Discrepancy in Ray Map channel ordering between Paper and Code implementation

Open
#218 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
6.3k
Forks
702
PR merge metrics
No merged PRs in 30d

Description

I have noticed a discrepancy regarding the definition of the 6-channel Ray Map (`camray`) between the paper description and the actual code implementation.

**1. Paper Definition**
In the paper, the definition of the camera ray is stated as follows:
> "For each pixel p, the camera ray r ∈ R6 is defined by its origin t ∈ R3 and direction d ∈ R3: r = (t, d)"

Image

This implies that the channel ordering is **[Origin, Direction]** (i.e., first 3 channels are $t$, last 3 channels are $d$).

**2. Code Implementation**
However, looking at `ray_utils.py` and `da3.py`, the implementation follows the order **[Direction, Origin]**.

**Evidence from `ray_utils.py`:**
In the `camray_to_caminfo()` function, the first 3 channels are used to solve for the rotation matrix (implying direction), while the last 3 channels are used to solve for translation (implying origin).

```python
# camray shape: (B, S, num_patches_y, num_patches_x, 6)

# Channels 0-3: Ray Direction
# Used for solving Rotation (R) and Intrinsics via RANSAC
ray_direction = camray[:, :, :3]

# Channels 3-6: Ray Origin
# Used for calculating Translation (T)
# The code calculates T by weighted averaging of the last 3 channels
T = torch.sum(camray[:, :, 3:] * confidence.unsqueeze(-1), dim=1) / torch.sum(confidence, ...)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.