google / google/tunix

ToolEnvironment crashes on explicit `finish` tool calls: arguments is a JSON string, not a dict

Open
#2,050 0 comments 0 reactions 1 assignee Claimed by @lc5211 View on GitHub
type:bug
Dominant language
Python
Stars
2.5k
Forks
345
Avg merge
1d 7h
Merged PRs (30d)
240

Description

## Summary

`ToolEnvironment._extract_llm_answer` raises `AttributeError: 'str' object has no attribute 'get'` when the model ends an episode with an **explicit** `finish` tool call (e.g. Qwen-format `{"name": "finish", "arguments": {"response": "..."}}`), instead of plain text.

## Root cause

The two code paths in `ToolAgent.update_from_model` produce **inconsistent types** for `function.arguments`:

- Fallback path (no tool call parsed → synthetic `finish`): `arguments` is a **dict** — `{"response": response}`.
- Parser path (tool call parsed from text): `arguments` is **`json.dumps`-stringified**:

```python
# tunix/rl/agentic/agents/tool_agent.py
args = tool_call.arguments
if isinstance(args, dict):
args = json.dumps(args)
```

`ToolEnvironment._extract_llm_answer` only handles the dict shape:

```python
# tunix/rl/agentic/environments/tool_environment.py
args = call["function"].get("arguments", {})
return args.get("response", "") # AttributeError when args is a JSON string
```

So any model that learned to emit `finish` through the documented tool-call format (rather than by answering in plain text) crashes the rollout. Note the episode-termination check just above (`name == "finish"`) matches fine — only the answer extraction breaks.

## Repro

Drive `TrajectoryCollectEngine` with a scripted `model_call` whose final turn returns:

```
{"name": "finish", "arguments": {"response": "The answer is 42."}}
```

with a `ToolAgent` (qwen parser) + `ToolEnvironment(reward_fn=...)` pair:

```
File ".../tunix/rl/agentic/environments/tool_environment.py", line 183, in _extract_llm_answer
return args.get("response", "")
^^^^^^^^
AttributeError: 'str' object has no attribute 'get'
```

Verified present at current `main`.

## Suggested fix

Make `_extract_llm_answer` tolerant of the stringified form:

```python
args = call["function"].get("arguments", {})
if isinstance(args, str):
try:
args = json.loads(args)
except json.JSONDecodeError:
return args
return args.get("response", "")
```

(or alternatively, stop stringifying arguments in `ToolAgent.update_from_model` — but other consumers may rely on the string form).

## Fix

Proposed fix with regression tests: https://github.com/google/tunix/pull/2049

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.