cloudflare / cloudflare/developer-platform

Workers AI: Tool call arguments produce broken control token instead of UUID across multiple models

Open
#54 3 comments 2 reactions 0 assignees View on GitHub
product:workers-ai
Dominant language
No language data
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Bug Report

### Description

When using the Workers AI OpenAI-compatible `/v1/chat/completions` endpoint with tool calling, models produce a broken control token `<|"|` in tool call arguments where a UUID string should appear. This occurs **identically across two unrelated models** — `@cf/google/gemma-4-26b-a4b-it` and `@cf/zai-org/glm-4.7-flash` — strongly suggesting the issue is in the Workers AI inference/serving layer, not in the models themselves.

**Routing the same model through OpenRouter (served by Parasail) with zero code changes produces correct UUIDs immediately.** Only the base URL and API key were changed.

### Reproduction

**Setup:** An agent running on a Durable Object calls the Workers AI API via the OpenAI-compatible endpoint. The agent has tools for looking up items (returning JSON with UUID fields) and performing actions using those IDs.

**Flow:**
1. Model calls a lookup tool → receives a JSON result containing valid UUIDs:
```json
{
"items": [
{"itemId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "name": "Item A", "price": 50},
{"itemId": "f9e8d7c6-b5a4-3210-fedc-ba9876543210", "name": "Item B", "price": 80}
]
}
```

2. Model calls an action tool → produces a broken argument:
```json
{"itemId": "<|\"|", "quantity": 1}
```

3. The model retries with the **exact same broken token** every time — it is deterministic, not random.

### Evidence this is a platform issue

| Provider | Model | UUID produced | Result |
|---|---|---|---|
| **Workers AI** | `@cf/google/gemma-4-26b-a4b-it` | `<\|"\|` | Broken |
| **Workers AI** | `@cf/zai-org/glm-4.7-flash` | `<\|"\|` | Broken (same token, different model) |
| **OpenRouter** (Parasail) | `google/gemma-4-26b-a4b-it` | `a1b2c3d4-e5f6-...` | Correct |

- **Two completely unrelated models produce the identical broken token** on Workers AI: Gemma 4 26B (Google, MoE) and GLM 4.7 Flash (Zhipu AI). Different architectures, tokenizers, and training data.
- The model correctly extracts and uses **other** string values from tool results (e.g., slugs are passed correctly). Only UUIDs in tool call arguments are affected.
- Gemma 4 26B scores **85.5% on τ2-bench** (agentic tool use benchmark) — it is highly capable at tool calling when served by other providers.

### Logs (redacted)

**Workers AI — Gemma 4 26B (`@cf/google/gemma-4-26b-a4b-it`):**
```json
{
"toolCalls": [
{"name": "list_items", "arguments": {}},
{"name": "lookup_item", "arguments": {"identifier": "some-item-slug"}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}}
]
}
```

**Workers AI — GLM 4.7 Flash (`@cf/zai-org/glm-4.7-flash`):**
```json
{
"toolCalls": [
{"name": "list_items", "arguments": {}},
{"name": "lookup_item", "arguments": {"identifier": "some-item-slug"}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}}
]
}
```

**OpenRouter (Parasail) — Same Gemma 4 26B model:**
```json
{
"toolCalls": [
{"name": "list_items", "arguments": {}},
{"name": "lookup_item", "arguments": {"identifier": "some-item-slug"}},
{"name": "perform_action", "arguments": {"itemId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "quantity": 1}}
]
}
```

### Environment

- **API endpoint:** `https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions` (OpenAI-compatible)
- **Models tested:** `@cf/google/gemma-4-26b-a4b-it`, `@cf/zai-org/glm-4.7-flash`
- **Client library:** `@mariozechner/pi-ai` v0.65.0 / `@mariozechner/pi-agent-core` v0.65.0 ([pi-mono](https://github.com/badlogic/pi-mono))
- **API protocol:** OpenAI-compatible `/v1/chat/completions` with tool definitions as JSON Schema (via TypeBox)
- **Runtime:** Cloudflare Worker with Durable Objects
- **Wrangler:** v4.71.0, `compatibility_date: 2026-03-12`, `compatibility_flags: ["nodejs_compat"]`
- **Worker placement:** `aws:us-east-2`
- **Date:** 2026-04-07

### Expected behavior

The model should reproduce the UUID from the tool result verbatim in subsequent tool call arguments.

### Actual behavior

The model emits a broken control token `<|\"|` — deterministic and identical across all models tested on Workers AI.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.