[Bug] Agent adapters rewrite malformed tool-call arguments before the client sees them
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 1.3k
- Avg merge
- 5h 36m
- Merged PRs (30d)
- 22
Description
Bug Description
At 16c15fc2068db1f73313f63f0d3c90e762b025d2, a recognized tool call with malformed JSON arguments is converted to {"_raw_arguments": raw} during parsing. The OpenAI adapter then JSON-serializes that sentinel, so the client never sees the argument string the model produced. The Anthropic adapter emits the same sentinel as an executable tool_use.input object.
This is externally observable adapter behavior rather than a reward or training-policy question:
- OpenAI's wire shape can carry the original argument string, including invalid JSON, and the client/harness can decide how to handle it.
- Anthropic's
tool_use.inputmust be an object, so malformed JSON cannot be represented faithfully as a tool use. Emitting a sentinel object makes a malformed model output look executable.
Steps to Reproduce
Run this in a normal slime development environment:
from slime.agent.adapters.anthropic import _build_reply_parts as anthropic_reply
from slime.agent.adapters.openai import _build_reply_parts as openai_reply
from slime.agent.parsing import ParsedModelOutput
raw = '{"cmd": "pytest"'
parsed = ParsedModelOutput(
reasoning="",
text="",
# This is the shape parse_tool_uses produces after JSONDecodeError.
tool_uses=[{"name": "bash", "input": {"_raw_arguments": raw}}],
)
openai_wire, _, _ = openai_reply(parsed, "stop")
anthropic_blocks, anthropic_stop, _ = anthropic_reply(parsed, "stop")
print(openai_wire["tool_calls"][0]["function"]["arguments"])
print(anthropic_blocks)
print(anthropic_stop)
The exact-main result is:
{"_raw_arguments": "{\"cmd\": \"pytest\""}
[{"type": "tool_use", ..., "name": "bash",
"input": {"_raw_arguments": "{\"cmd\": \"pytest\""}}]
tool_use
The original argument string was {"cmd": "pytest".
Expected Behavior
Recommended boundary behavior:
- Preserve the parser's raw argument string separately from its parsed object/error state.
- Project the raw string unchanged to OpenAI
function.arguments; do not parse and reserialize it for the wire response. - Use the parsed mapping only for chat-template/history rendering.
- For Anthropic, fail closed when arguments cannot be represented as an input object: do not advertise an executable
tool_use. A normal non-tool/text response would keep the session alive without fabricating valid input.
Would maintainers prefer that Anthropic fallback, or another explicit non-executable outcome?
Actual Behavior
parse_tool_usesreplaces invalid JSON with a sentinel mapping.- The OpenAI reply builder serializes that mapping instead of forwarding the model-produced argument string.
- The Anthropic reply builder emits the sentinel as a normal
tool_use.input.
Environment
- slime version:
16c15fc2068db1f73313f63f0d3c90e762b025d2(latestmainwhen reproduced) - Python version: 3.13.13
- PyTorch version: N/A; pure adapter reproduction
- CUDA/ROCm version: N/A
- GPU type and count: N/A
- OS: Linux
- SGLang version: N/A for the adapter-only reproduction; the linked parser branch produces the reproduced sentinel shape
- Megatron-LM version: N/A
Additional Context
A narrow bug-fix PR can preserve both raw and parsed arguments in the existing parsed tool-use record, project each at the appropriate boundary, and add CPU adapter regressions. It does not need to change sampled output token IDs, reward handling, or tool schema policy.
PR #2283 modifies the same OpenAI reply builder for parallel calls, but this issue is independent; any implementation can be rebased if that PR lands first.
Pre-submission Checklist
- I have read the CONTRIBUTING.md and understand the collaboration scope.
- I have searched existing issues and found no direct duplicate.
- I reproduced this against the latest public
main. - I have provided a minimal reproduction.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with slime/agent/parsing.py and the _build_reply_parts functions in slime/agent/adapters/openai.py and anthropic.py. Trace how parse_tool_uses stores malformed arguments, then add CPU adapter regressions. Done means OpenAI preserves the original argument string, while Anthropic does not advertise malformed input as executable tool_use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100