FoundationPose with custom object

For performing pose estimation on a novel object I understand that I need -

1. A object detection model trained on the new object. For this I used : rtdetr_r101vd which seems to be the same one that the sytheticaDETR in the quickstart is using. This was then converted to engine plan model.
   /usr/src/tensorrt/bin/trtexec --onnx=${ISAAC_ROS_WS}/isaac_ros_assets/models/custom/custom.onnx --saveEngine=${ISAAC_ROS_WS}/isaac_ros_assets/models/custom/custom.plan

2. A .obj file for the object. I dont have a uv texture map for the object so I removed that argument from the ros2 launch command.

   ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono_rect_depth,foundationpose mesh_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/custom/custom.obj score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan rt_detr_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/rtdetr/custom.plan

However I was not able to pose estimate on the new object. I am using realsense 435 for the project. I also tried detection alone using isaac ros rtdetr tutorial which also seems to be not working.

I had tested the onnx model seperately and it works. I guess this must be something to do with converting to tensorRT.