Agent does not see message argument in ToolMock
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 589
- Forks
- 76
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 3
Description
Describe the bug
Mock of PublishROS2MessageTool was created in PR https://github.com/RobotecAI/rai/pull/487.
Agent, when given this tool does not see message argument at all.
To Reproduce
Steps to reproduce the behavior:
- Go to branch
jm/feat/tool-benchmark-custom-interfaces - Run
python src/rai_bench/rai_bench/examples/tool_calling_agent_test_bench.py
- Check logs for agent output
Expected behavior
Agent should properly see all arguments
Screenshots
output of the agent when asked what arguments the tool has:
AIMessage(content='The `publish_ros2_message` tool has the following arguments:\n\n1. **topic**: A string that specifies the topic to which the message will be published.\n2. **message_type**: A string that represents the type of the message that will be published (e.g., `std_msgs/msg/String`). \n\nIf you need further assistance
When declared explicitly in the mock class and added new field to args_shema:
The agent outputs:
AIMessage(content='The `publish_ros2_message` tool has the following arguments:\n\n1. **topic**: The topic to which the message will be published (as a string).\n2. **message_type**: The type of the message being published (as a string).\n3. **costam**: This appears to be a placeholder for the actual message content that you want to publish (as a string).\n\nIf you need
So for some reason agent always ignores only message argument
When agent uses original tool, the problem does not occur:
Platform
- OS: Ubuntu 22.04
- ROS 2 Version - Humble
Version
Release number or commit hash.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run src/rai_bench/rai_bench/examples/tool_calling_agent_test_bench.py on the jm/feat/tool-benchmark-custom-interfaces branch and inspect the agent logs. Compare the PublishROS2MessageTool mock from PR 487 with the original tool and the explicitly declared mock example; done means the agent reports the message argument along with topic and message_type.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, robotics, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100