RobotecAI / RobotecAI/rai

Agent does not see message argument in ToolMock

Open
#492 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug invalid priority/minor
Dominant language
Python
Stars
589
Forks
76
Avg merge
1d 13h
Merged PRs (30d)
3

Description

Describe the bug
Mock of PublishROS2MessageTool was created in PR https://github.com/RobotecAI/rai/pull/487.
Agent, when given this tool does not see message argument at all.

To Reproduce
Steps to reproduce the behavior:

  1. Go to branch jm/feat/tool-benchmark-custom-interfaces
  2. Run
python src/rai_bench/rai_bench/examples/tool_calling_agent_test_bench.py
  1. Check logs for agent output

Expected behavior
Agent should properly see all arguments

Screenshots

Image

output of the agent when asked what arguments the tool has:

AIMessage(content='The `publish_ros2_message` tool has the following arguments:\n\n1. **topic**: A string that specifies the topic to which the message will be published.\n2. **message_type**: A string that represents the type of the message that will be published (e.g., `std_msgs/msg/String`). \n\nIf you need further assistance

When declared explicitly in the mock class and added new field to args_shema:

Image

The agent outputs:

AIMessage(content='The `publish_ros2_message` tool has the following arguments:\n\n1. **topic**: The topic to which the message will be published (as a string).\n2. **message_type**: The type of the message being published (as a string).\n3. **costam**: This appears to be a placeholder for the actual message content that you want to publish (as a string).\n\nIf you need

So for some reason agent always ignores only message argument

When agent uses original tool, the problem does not occur:

Image

Platform

  • OS: Ubuntu 22.04
  • ROS 2 Version - Humble

Version
Release number or commit hash.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run src/rai_bench/rai_bench/examples/tool_calling_agent_test_bench.py on the jm/feat/tool-benchmark-custom-interfaces branch and inspect the agent logs. Compare the PublishROS2MessageTool mock from PR 487 with the original tool and the explicitly declared mock example; done means the agent reports the message argument along with topic and message_type.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, robotics, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.