InternRobotics / InternRobotics/NavDP

[X-NavDP] How to launch full 56-scene training with 8 GPUs?

Open
#122 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
793
Forks
69
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for releasing X-NavDP!
Appendix B describes training with 56 scene processes, each running 8 environments, giving 448 parallel environments. Each process keeps its assigned scene and periodically switches embodiments. Paper
However, the README example launches 8 processes, and the public implementation uses:
torch.cuda.set_device(local_rank)
scene_index = rank % num_scenes
See GPU assignment and scene assignment.
My understanding is that, on one node with 8 GPUs, this command trains on only 8 fixed scenes. Increasing NPROC_PER_NODE to 16 or 56 would select GPU indices beyond the available devices, rather than automatically assigning multiple scene processes to each GPU.
Could you clarify:

  1. What hardware and process topology were used for the full 56-scene experiment: number of nodes, GPUs per node, total DDP ranks, and scene processes per GPU?
  2. If multiple scene processes shared a GPU, could you provide the corresponding GPU-assignment implementation and launch command?
  3. What is the recommended way to reproduce full 56-scene training on 8 GPUs? Could you also share the GPU model/memory capacity used and, if available, peak memory usage?
    If my reading of the implementation is incorrect, please point me to the relevant scheduling logic. An exact launch command/configuration for the Table 1 experiment would be very helpful.
    Thank you!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the README training section and the GPU and scene assignment logic in baselines/x-navdp/train.py around lines 138 and 225. Determine the experiment's process topology and resource requirements from the project materials, then document an exact launch command and configuration for reproducing 56-scene training on 8 GPUs. Done means the README explains the assignment behavior, hardware assumptions, and expected memory usage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
distributed-systems, documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.