charmbracelet / charmbracelet/fantasy
When validating tool parameters an improved error message could encourage LLMs to auto-heal their tool call
- Dominant language
- Go
- Stars
- 987
- Forks
- 137
- Avg merge
- 14h 41m
- Merged PRs (30d)
- 12
Description
### Description
An example is Crush's edit tool. Even frontier LLMs will switch between `old_string` (correct) and `old-string` (incorrect) the current error message simply states `ERROR: missing required parameter: old_string` but if the error string said something like `ERROR: missing required parameter: old_string, unknown additional parameter old-string` via stricter checking against declared parameters. Then most LLM's would notice the problem rather than assume they performed the tool-use correctly and it's the tool that is broken.
Contributor guide
Research direction
Start by locating the tool parameter validation entry point and inspect how declared and unknown parameters are currently handled. Use Crush's edit tool example with old_string and old-string to reproduce the error, then add or update validation coverage so the message identifies both the missing required parameter and the unknown additional parameter.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100