charmbracelet / charmbracelet/fantasy

When validating tool parameters an improved error message could encourage LLMs to auto-heal their tool call

Open
#304 0 comments 2 reactions 0 assignees View on GitHub
area: tools
Dominant language
Go
Stars
987
Forks
137
Avg merge
14h 41m
Merged PRs (30d)
12

Description

### Description

An example is Crush's edit tool. Even frontier LLMs will switch between `old_string` (correct) and `old-string` (incorrect) the current error message simply states `ERROR: missing required parameter: old_string` but if the error string said something like `ERROR: missing required parameter: old_string, unknown additional parameter old-string` via stricter checking against declared parameters. Then most LLM's would notice the problem rather than assume they performed the tool-use correctly and it's the tool that is broken.

Contributor guide

Open the contributing guide

Research direction

Start by locating the tool parameter validation entry point and inspect how declared and unknown parameters are currently handled. Use Crush's edit tool example with old_string and old-string to reproduce the error, then add or update validation coverage so the message identifies both the missing required parameter and the unknown additional parameter.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ai
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.