huggingface / huggingface/smolagents
Make system prompt examples consistent with the agent’s actual logs
- Dominant language
- Python
- Stars
- 29.3k
- Forks
- 3k
- Avg merge
- 17m
- Merged PRs (30d)
- 2
Description
I’ve noticed a small discrepancy that might affect the model’s performance. In the system prompt examples ([code_agent.yaml](https://github.com/huggingface/smolagents/blob/main/src/smolagents/prompts/code_agent.yaml)), each step’s output is shown simply as:
```
Observation:
...
```
However, in actual usage the agent output is more detailed, including lines such as:
```
Call id: call_...
Observation:
Execution logs: ...
Last output from code snippet:
None
```
This difference can slightly reduce clarity for the large language model, because it doesn’t consistently see the same format it was “taught” to use in the system prompt examples.
**Suggestion**
Two possible approaches:
1. **Update the system prompt examples** to show the additional lines, including `Call id`, `Execution logs`, etc.
2. **Simplify the actual agent output** so it only presents the simpler “Observation” style, exactly as in the system prompt examples.
Either option would help the LLM follow the desired output style more reliably.
Contributor guide
Research direction
Start with src/smolagents/prompts/code_agent.yaml and compare its step examples with the runtime format shown in the issue. First clarify whether the prompt or the agent output is the intended source of truth, then update the selected side and verify that the examples and actual logs use the same format.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100