RobotecAI / RobotecAI/rai

MultiThreadedExecutor in ROS2ARIConnector slows things down 100 times

Open
#513 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug priority/minor
Dominant language
Python
Stars
589
Forks
76
Avg merge
1d 13h
Merged PRs (30d)
3

Description

related: https://github.com/ros2/rclpy/issues/1223

Describe the bug
I was reviewing this lovely PR which transforms our old openset detection and segmentation solution into agents.
I've noticed a critically bad performance and decided to dive into debugging.
Tried few things:

  1. substitution wait_for_shutdown with time.sleep (script below)
  2. Debugging weights loading (GroundedSamAgent, GroundingDinoAgent)
  3. Finally decided to change the executor in ROS2ARIConnector SingleThreaded. The speed. The performance. The speed. The power.

Image

To Reproduce
Steps to reproduce the behavior:

  1. Have RAI set up (colcon build + poetry install --with openset + source setup_shell.sh)
  2. git checkout 9cf286f
  3. Run the following script
from rai_open_set_vision.agents import GroundingDinoAgent, GroundedSamAgent
from rai.utils import wait_for_shutdown
import rclpy


def main():
    rclpy.init()
    agent1 = GroundingDinoAgent()
    agent2 = GroundedSamAgent()
    agent1.run()
    agent2.run()
    wait_for_shutdown([agent1, agent2])
    rclpy.shutdown()


if __name__ == "__main__":
    main()
  1. Run these tests

from rai.communication.ros2 import ROS2ARIConnector, ROS2ARIMessage
from rai.utils import ROS2Context
from rai_interfaces.srv import RAIGroundingDino, RAIGroundedSam
import cv2
import cv_bridge

from rai.tools.ros2.utils import ros2_message_to_dict

@ROS2Context()
def test_grounding_dino_agent():
    connector = ROS2ARIConnector()
    bridge = cv_bridge.CvBridge()

    image = bridge.cv2_to_imgmsg(cv2.imread("docs/imgs/o3deSimulation.png"))

    msg = RAIGroundingDino.Request()
    msg.source_img = image
    msg.classes = ["chair"]
    msg.box_threshold = 0.5
    msg.text_threshold = 0.5
    
    ari_msg = ROS2ARIMessage(payload=ros2_message_to_dict(msg))
    response = connector.service_call(target="/grounding_dino_classify", message=ari_msg, msg_type="rai_interfaces/srv/RAIGroundingDino", timeout_sec=10)
    print(response.payload)

@ROS2Context()
def test_grounded_sam_agent():
    connector = ROS2ARIConnector()
    bridge = cv_bridge.CvBridge()

    image = bridge.cv2_to_imgmsg(cv2.imread("docs/imgs/o3deSimulation.png"), encoding="rgb8")

    msg = RAIGroundedSam.Request()
    msg.source_img = image
    msg.detections = {}

    ari_msg = ROS2ARIMessage(payload=ros2_message_to_dict(msg))
    response = connector.service_call(target="/grounded_sam_segment", message=ari_msg, msg_type="rai_interfaces/srv/RAIGroundedSam", timeout_sec=10)
    print(response.payload)
  1. Note the time it took pytest to finish
  2. Change the following line to SingleThreadedExecutor (remember to update import) https://github.com/RobotecAI/rai/blob/0675ff6036bbd1450f34426f10d64f999ab91777/src/rai_core/rai/communication/ros2/connectors/ari_connector.py#L106
  3. Run the script and pytest again, note the time.
  4. Should be 100 times faster.

Changing the num_threads parameter in MultiThreadedExecutor did not result in speed up.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Platform

  • OS: 24.04
  • ROS 2 Jazzy

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/rai_core/rai/communication/ros2/connectors/ari_connector.py at line 106 and review how ROS2ARIConnector selects its executor. Reproduce the issue using the provided GroundingDinoAgent and GroundedSamAgent script and service-call tests, then compare execution times with the two executor choices. Done means the reported slowdown is resolved and the reproduction completes without regressions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, robotics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.