huggingface / huggingface/smolagents

[Feature request] Enable Tool arguments to process pydantic schemas properly

Open
#699 3 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
29.3k
Forks
3k
Avg merge
17m
Merged PRs (30d)
2

Description

**Is your feature request related to a problem? Please describe.**
Tools don't properly pass pydantic schema for the objects in their functions.
For example, my tool takes the following as its input parameter
```
class ReportType(Enum):
REPORT_1 = "report_1"
REPORT_2 = "report_2"

@classmethod
def values(cls) -> list[str]:
return [type.value for type in cls]

class ReportConfig(BaseModel):
sheet_name: str = Field(description="Name of the sheet")
requested_start_date: date = Field(description="Start date")
requested_end_date: date = Field(description="End date")
report_type: str = Field(
description="Type of the report",
enum=ReportType.values()
)
```

If I call
`ReportConfig.model_json_schema()` to be passed into, say, OpenAI,
```
tools=[
{
"type": "function",
"function": {
"name": "report_work",
"description": "Adds a report. The configuration must follow the ReportConfig schema.",
"parameters": ReportConfig.model_json_schema(),
},
}
]
```
I get a correct call
`ReportConfig(sheet_name='ABC', requested_start_date=datetime.date(2024, 1, 1), requested_end_date=datetime.date(2024, 1, 31), report_type='report_1')`
with the correct tool call
`{'properties': {'sheet_name': {'description': 'Name of the sheet',
'title': 'Sheet Name',
'type': 'string'},
'requested_start_date': {'description': 'Start date',
'format': 'date',
'title': 'Requested Start Date',
'type': 'string'},
'requested_end_date': {'description': 'End date',
'format': 'date',
'title': 'Requested End Date',
'type': 'string'},
'report_type': {'description': 'Type of the report',
'enum': ['report_1', 'report_2'],
'title': 'Report Type',
'type': 'string'}},
'required': ['sheet_name',
'requested_start_date',
'requested_end_date',
'report_type'],
'title': 'ReportConfig',
'type': 'object'}`

However, when I use `LiteLLMModel` with a tool with the following inputs:
inputs = {
"report_config": {
"type": "object",
"description": "Configuration for the new report.",
"properties": ReportConfig.model_json_schema()['properties'],
},
}

I get the call with
`{'report_config': {'sheet_name': 'ABC',
'requested_start_date': '2024-01-01',
'requested_end_date': '2024-01-31'}}`

So first, `required` fields are not passed, which results in an incomplete request. Moreover, the `enum` is not properly passed as `get_json_schema` function in `..._hint_utils.py` actually uses strange `choice` way of defining enums

**Describe the solution you'd like**
I want to be able to specify `ReportConfig.model_json_schema()` in my tool input as a parameter, for a complex nested model. This will ensure much more consistent tool calling

**Is this not possible with the current options.**
At least I am not aware how to pass the schema differently

**Describe alternatives you've considered**
The only alternative I see is to write my own Tool class and overwrite most of the functions there, but it's possible this won't be enough as the LiteLLMModel might work with tools in a very specific way.

**Additional context**
Basically, it would be good to have at least parity with openai/anthropic tool calls capabilities in tool arguments

Contributor guide

Open the contributing guide

Research direction

Start with get_json_schema in ..._hint_utils.py and follow how LiteLLMModel converts tool inputs into schemas. Compare that conversion with the nested ReportConfig.model_json_schema() shown in the issue. Done means required fields and enum values are preserved for nested model arguments and appear correctly in the resulting tool call.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.