How to get real depth values from approximated depth in certain ray.
- Dominant language
- Jupyter Notebook
- Stars
- 2.8k
- Forks
- 458
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I'm appreciate to your great work!
I'm implementing some light-field based neural network which have different input structure for MLP.
The MLP outputs the rgb color and depth values which are in range [0, 1] in NDC coordinates by sigmoid function to corresponding ray.
However, my questions is follows
1. For NDC coordinates, if I correctly understand, the scene bounded in `near(0)` and `far(1)` plane. Then, do all components exist between near and far plane?
2. As we can see the `get_rays` functions, **z** value is instantiate with `-1`. I can sure that camera origin is world coordinates from the evidence that `c2w[:3,-1]` part in following code. As I understand, the rays are still in world coordinate since the magnitude of direction vector doesn't matter. Am I right?

3. I get some depth value and compute the corresponding points using follow code. I intend the code works regardless of the coordinate system, NDC or world. z_vals contains `N_rays x [near, far]` values. Is this code right..?

4. If points are correctly computed in given ray with some depth values, how to compute the real depth value when NDC coordinate system is used?
For example, if I have P(Xp, Yp, Zp) in NDC coordinates, following code can back project to real depth value?
I use same coordinate system to NeRF, (right, up, backward), in world and I guess NDC coordinate system is (right, up, forward).
I read about the [issue](https://github.com/bmild/nerf/issues/35) you raised in NeRF original GitHub, but I think it still not work properly for my code.
For given `ndc_pts`, which have `N_rays x (Xp, Yp, Zp)`, we back-project `ndc_pts` to world following your issue.

But what I found in your GitHub [issue](https://github.com/kwea123/nerf_pl/issues/48), which refer the necessary of back project with `K^-1 * d * (x y 1)`.

For depth direction, I think `depth_real = 1/(1-depth_ndc)` is different because of the different coordinate system.
Then, does `K^-1 * d * (x y 1)` mean same process with the mentioned code from NeRF author?(which I used).
Thanks to read my questions.
If you reply to this issue, it will be really helpful...!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the get_rays function and the NDC coordinate calculations shown in the issue, then compare them with the linked NeRF and nerf_pl discussions. Resolve how ray points and depth should be transformed between NDC and world coordinates, and document the correct real-depth calculation for the stated coordinate conventions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100