InternRobotics / InternRobotics/NavDP
About the calculation formula for SPL
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 793
- Forks
- 69
- PR merge metrics
- No merged PRs in 30d
Description
In the current evaluation code for NavDP/X-NavDP, the reference distance used for SPL is the 2D Euclidean distance between the start and goal points, rather than the geodesic shortest path (which, in most cases, corresponds to the A* shortest path). However, the SPL formula defined in the 2018 paper "On Evaluation of Embodied Navigation Agents" (arXiv:1807.06757) explicitly specifies the use of the geodesic shortest path in the numerator. I would like to know why the formula used here differs from that definition.
This is a matter of detail; I would be extremely grateful if you could answer this for me.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the evaluation code for NavDP/X-NavDP alongside the cited 2018 SPL paper. Identify where the reference distance is calculated and compare the implemented 2D Euclidean distance with the paper's geodesic definition. Done means documenting a maintainer-confirmed explanation for the discrepancy or clarifying which formula should be used.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100