ByteDance-Seed / ByteDance-Seed/Depth-Anything-3

What evaluation resolution and split were used for the KITTI AbsRel metric in Table 11?

Open
#252 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.3k
Forks
702
PR merge metrics
No merged PRs in 30d

Description

Thank you for the excellent work on Depth Anything V3! I have a question regarding the metric depth evaluation reported in Table 11 of the paper.

In Table 11, you report AbsRel and δ₁ for the DA3-metric model on the KITTI benchmark (AbsRel = 0.086, δ₁ = 0.953). However, the paper does not explicitly specify:

Input resolution during evaluation: What resolution was used when running inference on KITTI test images? Was the full original resolution used, or was the input resized/cropped (e.g., to match the training base resolution of 504)?
Evaluation split: Which KITTI split was used for evaluation — the official KITTI raw split, the Eigen split, or some other split? Was the evaluation protocol following [Eigen et al., 2014] or a different protocol?
Evaluation crop: Was the standard KITTI evaluation crop applied (as commonly used in prior work, e.g., cropping to the region specified by gt_predicted masks)?
These details are important for fair comparison and reproducibility, as different evaluation protocols and resolutions can lead to significantly different AbsRel values. For example, some methods report KITTI AbsRel evaluated at the original resolution with the standard crop, while others use downsampled inputs.

Any clarification would be greatly appreciated!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the paper's Table 11 and the KITTI evaluation details referenced in the issue. Confirm the inference resolution, evaluation split and protocol, and whether the standard KITTI crop was applied; done means documenting these three details clearly enough to reproduce the reported AbsRel and δ₁ values.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.