huggingface / huggingface/smolagents

Make system prompt examples consistent with the agent’s actual logs

Open
#830 0 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
29.3k
Forks
3k
Avg merge
17m
Merged PRs (30d)
2

Description

I’ve noticed a small discrepancy that might affect the model’s performance. In the system prompt examples ([code_agent.yaml](https://github.com/huggingface/smolagents/blob/main/src/smolagents/prompts/code_agent.yaml)), each step’s output is shown simply as:
```
Observation:
...
```
However, in actual usage the agent output is more detailed, including lines such as:
```
Call id: call_...
Observation:
Execution logs: ...
Last output from code snippet:
None
```
This difference can slightly reduce clarity for the large language model, because it doesn’t consistently see the same format it was “taught” to use in the system prompt examples.

**Suggestion**

Two possible approaches:
1. **Update the system prompt examples** to show the additional lines, including `Call id`, `Execution logs`, etc.
2. **Simplify the actual agent output** so it only presents the simpler “Observation” style, exactly as in the system prompt examples.

Either option would help the LLM follow the desired output style more reliably.

Contributor guide

Open the contributing guide

Research direction

Start with src/smolagents/prompts/code_agent.yaml and compare its step examples with the runtime format shown in the issue. First clarify whether the prompt or the agent output is the intended source of truth, then update the selected side and verify that the examples and actual logs use the same format.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.