NVIDIA-ISAAC-ROS / NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation

Camera calibration parameters

Open
#36 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
501
Forks
57
PR merge metrics
No merged PRs in 30d

Description

I premise that I compiled the package by using "colcon build --packages-select isaac_ros_dope".
I'm facing some problems with this package while running on a NVIDIA Jetson Orin Nano.
More in detail, I think there could be some issues with the intrinsics of the camera.
I'm using the package with the images provided by an Intel RealSense d435i that is configured with "rgb_camera.profile:=640x480x30".
I'm quite sure that the node "dope_decoder_node" is loading the camera calibration parameters that I specified in the "dope_config.yaml" file.
The calibration parameters specified in the "dope_config.yaml" are the following: [605.6713256835938, 0, 329.4186706542969, 0, 605.2176513671875, 245.733154296875, 0, 0, 1]. These parameters are the same of those in the "camera_info" topic.
The package is running quite well in the Jetson Orin Nano but the estimated position of the object that I'm detecting (Campbell's Tomato soup) looks too close to the camera.
The object is approximately 20 cm far from the camera but DOPE is estimating a distance of something like 10 cm.
Moreover, I recorded a bag file with the following topics: images, camera_info, and poses. In my laptop, I converted the bag file to be used in ROS1 Noetic. Subsequently, I used the bag file to run DOPE (https://github.com/NVlabs/Deep_Object_Pose) and compare the results.
As you can see from the following image, the results provided by the isaac_ros_dope and the ROS1 DOPE are different.

dope_results_comparison

Thank you in advance for your help.

PS. In the Jetson Orin Nano, I'm using isaac_ros_dope with the weights converted in the ONNX format and with the "isaac_ros_dope_tensor_rt.launch.py" file.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with dope_config.yaml, dope_decoder_node, and isaac_ros_dope_tensor_rt.launch.py, then reproduce the reported difference using the recorded images, camera_info, and poses topics. Compare the Isaac ROS results with ROS1 DOPE using the same bag data and calibration parameters; done means the source of the distance discrepancy is isolated and the expected pose behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-vision, robotics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.