InternRobotics / InternRobotics/NavDP
[BUG] Memory Overflow and Thread Error in `MWF4WLIKTIFZIAABAAAAADY8_usd` Scene
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 793
- Forks
- 69
- PR merge metrics
- No merged PRs in 30d
Description
Description:
I encountered an issue when testing the NavDP model with the MWF4WLIKTIFZIAABAAAAADY8_usd scene in scenes_commercial. The memory usage spikes to 80GB during the test, whereas other scenes only use around 10GB. After this, the following errors appear:
2025-11-20 08:11:29 [1,286,639ms] [Warning] [omni.ujitso] UJITSO : Build storage validation failed for Storage status: 1
http://localhost:8002/navigator_reset
No trajectory available, using zero action
No trajectory available, using zero action
No trajectory available, using zero action [ERROR:0@1278.827] global parallel_impl.cpp:244 WorkerThread 54: Can't spawn new thread: res[0/1840]
LLVM ERROR: out of memory
[ERROR:0@1278.827] global parallel_impl.cpp:244 WorkerThread 55: Can't spawn new thread: res = 11
[ERROR:0@1278.829] global parallel_impl.cpp:244 WorkerThread 56: Can't spawn new thread: res = 11
[ERROR:0@1278.829] global parallel_impl.cpp:244 WorkerThread 57: Can't spawn new thread: res = 11
[ERROR:0@1278.829] global parallel_impl.cpp:244 WorkerThread 58: Can't spawn new thread: res = 11
[ERROR:0@1278.829] global parallel_impl.cpp:244 WorkerThread 59: Can't spawn new thread: res = 11
LLVM ERROR: out of memory
Aborted (core dumped)
Steps to Reproduce:
-
Clone the NavDP repository and set up the environment.
-
Download and set up the
scenes_commercial/MWF4WLIKTIFZIAABAAAAADY8_usdscene -
Run the model with the
scenes_commercial/MWF4WLIKTIFZIAABAAAAADY8_usdscene using the following command:python eval_pointgoal_wheeled.py --port 8000 --scene_dir assets/scenes/internscenes_commercial/scenes_commercial --scene_index 17 --scene_scale 0.01
The modefied part of eval_pointgoal_wheeled.py is as follows:
all_scenes = sorted(os.listdir(args_cli.scene_dir))
scene_path = os.path.join(args_cli.scene_dir,all_scenes[args_cli.scene_index]) + "/"
- Observe the error and the program aborts.
Additional Information:
- This issue only occurs in the
MWF4WLIKTIFZIAABAAAAADY8_usdscene. Other scenes, like those in thecluttered_easyorinternscenes_commercialdirectories, do not show this behavior. - It appears to be related to the memory management or threading during the model's operation.
Environment:
- Operating System: Ubuntu 22.04.5 LTS
- GPU: NVIDIA GeForce RTX 4090
- Memory: 1 TB
Request:
Could you please advise on possible solutions for this issue? Any suggestions for addressing this issue would be appreciated.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with eval_pointgoal_wheeled.py and reproduce the command using scene_index 17 and scene_scale 0.01. Compare memory and thread behavior with the unaffected scenes, then trace the scene-specific failure shown in the logs; done means identifying the cause and providing a verified fix or workaround for MWF4WLIKTIFZIAABAAAAADY8_usd.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, robotics
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100