Consistency between per-pixel rays and global camera parameters

Hi, thanks for the great work!

I had a small confusion while reading the paper: the model predicts both per-pixel ray maps and global camera parameters (f, q, t) separately, and as far as I can tell, there’s no loss enforcing consistency between them. Since both encode camera geometry, they’re somewhat redundant—but likely not perfectly aligned in practice.

So I’m wondering: for downstream tasks like point cloud reconstruction, which one should we actually use?

Should 3D points be computed from depth + ray, or from depth + (f, q, t)? The two will generally give different results, so it’s unclear which is considered the “main” output.

Thanks in advance for clarifying!