InternRobotics / InternRobotics/NavDP

Clarification: does the LogoPlanner startgoal Isaac simulation evaluation use simulator GT pose for MPC tracking?

Open
#112 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
793
Forks
69
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for releasing LoGoPlanner and the NavDP benchmark.

I want to clarify the simulation evaluation protocol. The paper/project page describes LoGoPlanner as using implicit localization without an
external localization/odometry module. From the released code, I agree that the LoGoPlanner policy/server itself does not seem to take GT pose
as input.

Policy/server side

logoplanner_server.py only receives goal/RGB/depth and returns trajectories:

goal_data = json.loads(request.form.get('goal_data'))
image_file = request.files['image']
depth_file = request.files['depth']

execute_trajectory, all_trajectory, all_values, trajectory_mask, sub_pointgoal_pd = \
    logoplanner_navigator.step_pointgoal(goal, image, depth)

return jsonify({
    'trajectory': execute_trajectory.tolist(),
    'all_trajectory': all_trajectory.tolist(),
    'all_values': all_values.tolist(),
    'sub_pointgoal_pd': sub_pointgoal_pd.tolist()
})

Isaac simulation evaluator side

In eval_startgoal_wheeled.py, the low-level MPC tracker appears to use simulator GT pose as feedback. x0 is built from Isaac camera/root state:

camera_pos = env.unwrapped.scene.sensors['camera_sensor'].data.pos_w.cpu().numpy()
camera_rot_quat = env.unwrapped.scene.sensors['camera_sensor'].data.quat_w_world.cpu().numpy()
camera_rot = R.from_quat(camera_rot_quat[:, [1, 2, 3, 0]]).as_matrix()

robot_vel = env.unwrapped.scene.articulations['robot'].data.root_lin_vel_w[0, :2].norm().cpu().numpy()
robot_ang_vel = env.unwrapped.scene.articulations['robot'].data.root_ang_vel_w[0, 2].cpu().numpy()

x0 = np.stack([
    camera_pos[:, 0],
    camera_pos[:, 1],
    np.arctan2(camera_rot[:, 1, 0], camera_rot[:, 0, 0]),
    [robot_vel],
    [robot_ang_vel],
], axis=-1)

Then this state is passed to MPC:

opt_u_controls, opt_x_states = mpc.solve(x0[i, :3])

And tracking_utils.py sets it as the MPC initial state:

self.opti.set_value(self.opt_x0, x00)

My understanding

  1. LoGoPlanner policy/server does not use GT pose as input.
  2. The released Isaac simulation evaluator does use simulator GT pose for low-level MPC tracking.
  3. This GT feedback may correct tracking/odometry errors during simulation, even though it is not part of the policy input.

Could you clarify whether the reported simulation results use this GT-pose MPC feedback?
Thanks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read eval_startgoal_wheeled.py and tracking_utils.py, focusing on how simulator camera and robot state are assembled into the MPC initial state. Compare that path with logoplanner_server.py's inputs, then document whether the reported simulation evaluation uses GT-pose feedback for MPC tracking.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
robotics
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.