huggingface / huggingface/smolagents

BUG: Tool schema validation is disabled under `python -O`

Open
#2,736 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
29.4k
Forks
3k
Avg merge
17m
Merged PRs (30d)
2

Description

### System info

- Repository: `huggingface/smolagents` (current `main`)
- Python optimization mode: `python -O`

### Information

`Tool.validate_arguments()` currently uses `assert` for several runtime validation checks, including validating the structure of `inputs`, `output_type`, and consistency between declared inputs and the `forward()` signature/schema.

Python removes `assert` statements when the interpreter runs with optimization enabled (`-O` or `-OO`). As a result, malformed `Tool` definitions that are rejected in normal execution can bypass parts of tool schema validation in optimized mode.

### Reproduction

```python
from smolagents import Tool

class BrokenTool(Tool):
name = "broken_tool"
description = "A deliberately invalid tool."
inputs = {
"query": {
"type": "string",
# Intentionally missing "description"
}
}
output_type = "string"

def forward(self, query: str) -> str:
return query

tool = BrokenTool()
print("Tool initialized:", tool)
```

Run normally:

```bash
python repro_tool_validation.py
```

The invalid tool is rejected by validation with an `AssertionError` because the input schema is missing `description`.

Run with optimization enabled:

```bash
python -O repro_tool_validation.py
```

The assertion-based validation is removed, allowing the malformed tool to initialize successfully.

### Expected behavior

Runtime validation of a `Tool` definition should behave consistently regardless of whether Python optimization mode is enabled. Invalid tool schemas should fail fast under both normal execution and `python -O` / `python -OO`.

### Possible fix

Replace assertion-based runtime validation in `Tool.validate_arguments()` with explicit exceptions such as `TypeError` / `ValueError`, preserving the existing validation semantics and error messages as closely as possible.

A regression test could run a minimal invalid tool definition in a subprocess using `sys.executable -O` to verify that validation remains active in optimized mode.

This is similar in failure mode to #2456, but affects `Tool.validate_arguments()` rather than `final_answer_checks`.

I'm happy to work on this issue and submit a PR if the approach is accepted.

Contributor guide

Open the contributing guide

Research direction

Start at Tool.validate_arguments() and use the repro_tool_validation.py example to trace the schema checks that currently rely on assert. Run the invalid tool under both normal Python and python -O, then add a subprocess regression test covering optimized execution. Done means malformed inputs still fail fast in both modes with explicit exceptions and preserved validation semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.