huggingface / huggingface/smolagents
BUG: Tool schema validation is disabled under `python -O`
- Dominant language
- Python
- Stars
- 29.4k
- Forks
- 3k
- Avg merge
- 17m
- Merged PRs (30d)
- 2
Description
### System info
- Repository: `huggingface/smolagents` (current `main`)
- Python optimization mode: `python -O`
### Information
`Tool.validate_arguments()` currently uses `assert` for several runtime validation checks, including validating the structure of `inputs`, `output_type`, and consistency between declared inputs and the `forward()` signature/schema.
Python removes `assert` statements when the interpreter runs with optimization enabled (`-O` or `-OO`). As a result, malformed `Tool` definitions that are rejected in normal execution can bypass parts of tool schema validation in optimized mode.
### Reproduction
```python
from smolagents import Tool
class BrokenTool(Tool):
name = "broken_tool"
description = "A deliberately invalid tool."
inputs = {
"query": {
"type": "string",
# Intentionally missing "description"
}
}
output_type = "string"
def forward(self, query: str) -> str:
return query
tool = BrokenTool()
print("Tool initialized:", tool)
```
Run normally:
```bash
python repro_tool_validation.py
```
The invalid tool is rejected by validation with an `AssertionError` because the input schema is missing `description`.
Run with optimization enabled:
```bash
python -O repro_tool_validation.py
```
The assertion-based validation is removed, allowing the malformed tool to initialize successfully.
### Expected behavior
Runtime validation of a `Tool` definition should behave consistently regardless of whether Python optimization mode is enabled. Invalid tool schemas should fail fast under both normal execution and `python -O` / `python -OO`.
### Possible fix
Replace assertion-based runtime validation in `Tool.validate_arguments()` with explicit exceptions such as `TypeError` / `ValueError`, preserving the existing validation semantics and error messages as closely as possible.
A regression test could run a minimal invalid tool definition in a subprocess using `sys.executable -O` to verify that validation remains active in optimized mode.
This is similar in failure mode to #2456, but affects `Tool.validate_arguments()` rather than `final_answer_checks`.
I'm happy to work on this issue and submit a PR if the approach is accepted.
Contributor guide
Research direction
Start at Tool.validate_arguments() and use the repro_tool_validation.py example to trace the schema checks that currently rely on assert. Run the invalid tool under both normal Python and python -O, then add a subprocess regression test covering optimized execution. Done means malformed inputs still fail fast in both modes with explicit exceptions and preserved validation semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100