NVlabs / NVlabs/SimFoundry

DROID background asset contains a baked-in static robot, causing duplicate arms

Open
#8 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
372
Forks
26
Avg merge
2m
Merged PRs (30d)
2

Description

Description

The bundled DROID scene background contains a static robot arm, while OmniGibson also instantiates the controllable simulated robot (robot0). This produces two visually overlapping robot arms: one remains completely static and the other moves with the simulation.

This appears to be a background asset / real-to-sim preprocessing issue rather than a duplicate robot instance in the scene registry.

Steps to reproduce

From the repository root, run:

python scripts/pipeline/C_application/stages/0_smoke_random_actions.py \
  --scene-json "$(pwd)/assets/scenes/DROID/droid_desk_serve_fruits/droid_desk_serve_fruits_scene_state_latest.json" \
  --external-sensors-config nv_franka_droid_high_res \
  --n-steps 20

Inspect either external-camera view in the generated smoke-test video.

The same behavior is visible with the standard nv_franka_droid camera configuration; high resolution only makes it easier to see.

Actual behavior

  • The simulated robot0 moves in response to random actions.
  • A second robot arm remains fixed in the reconstructed background throughout the video.
  • In a first/middle/last-frame comparison, the background arm keeps exactly the same pose while the simulated arm changes pose.

Expected behavior

The reconstructed background should be robot-free, so only the controllable simulated robot is visible.

Asset-level diagnosis

The scene JSON declares only one robot, robot0. It also loads:

mesh_background_0
  usd_path: ../../../backgrounds/mesh_backgrounds/droid_v1.usd
  fixed_base: true

droid_v1.usd references droid_v1.usdz. The static arm appears to be baked into this mesh/texture reconstruction. Adding the simulated robot on top therefore creates the duplicate-arm artifact.

A robot mask/inpainting pass before background reconstruction, or a robot-free replacement for droid_v1.usdz, should resolve the issue.

Environment

  • SimFoundry commit: 9e34ebefcd02
  • Ubuntu 24.04.4
  • NVIDIA GeForce RTX 5090
  • NVIDIA driver 580.173.02
  • Smoke test completed successfully (20 steps, H.264 output)

I can provide the generated comparison frames or video if useful.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with assets/scenes/DROID/droid_desk_serve_fruits/droid_desk_serve_fruits_scene_state_latest.json and inspect the referenced backgrounds/mesh_backgrounds/droid_v1.usd and droid_v1.usdz assets. Reproduce the issue with scripts/pipeline/C_application/stages/0_smoke_random_actions.py and compare the external-camera frames. Done means the reconstructed background contains no static robot arm, leaving only controllable robot0 visible.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-graphics, robotics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.