cloudflare / cloudflare/developer-platform

Workers AI: Tool call arguments produce broken control token instead of UUID across multiple models

Ouverte
#54 3 commentaires 2 réactions 0 personnes assignées Voir sur GitHub
product:workers-ai
Langage dominant
Aucune donnée de langage
Étoiles
1
Forks
0
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

## Bug Report

### Description

When using the Workers AI OpenAI-compatible `/v1/chat/completions` endpoint with tool calling, models produce a broken control token `<|"|` in tool call arguments where a UUID string should appear. This occurs **identically across two unrelated models** — `@cf/google/gemma-4-26b-a4b-it` and `@cf/zai-org/glm-4.7-flash` — strongly suggesting the issue is in the Workers AI inference/serving layer, not in the models themselves.

**Routing the same model through OpenRouter (served by Parasail) with zero code changes produces correct UUIDs immediately.** Only the base URL and API key were changed.

### Reproduction

**Setup:** An agent running on a Durable Object calls the Workers AI API via the OpenAI-compatible endpoint. The agent has tools for looking up items (returning JSON with UUID fields) and performing actions using those IDs.

**Flow:**
1. Model calls a lookup tool → receives a JSON result containing valid UUIDs:
```json
{
"items": [
{"itemId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "name": "Item A", "price": 50},
{"itemId": "f9e8d7c6-b5a4-3210-fedc-ba9876543210", "name": "Item B", "price": 80}
]
}
```

2. Model calls an action tool → produces a broken argument:
```json
{"itemId": "<|\"|", "quantity": 1}
```

3. The model retries with the **exact same broken token** every time — it is deterministic, not random.

### Evidence this is a platform issue

| Provider | Model | UUID produced | Result |
|---|---|---|---|
| **Workers AI** | `@cf/google/gemma-4-26b-a4b-it` | `<\|"\|` | Broken |
| **Workers AI** | `@cf/zai-org/glm-4.7-flash` | `<\|"\|` | Broken (same token, different model) |
| **OpenRouter** (Parasail) | `google/gemma-4-26b-a4b-it` | `a1b2c3d4-e5f6-...` | Correct |

- **Two completely unrelated models produce the identical broken token** on Workers AI: Gemma 4 26B (Google, MoE) and GLM 4.7 Flash (Zhipu AI). Different architectures, tokenizers, and training data.
- The model correctly extracts and uses **other** string values from tool results (e.g., slugs are passed correctly). Only UUIDs in tool call arguments are affected.
- Gemma 4 26B scores **85.5% on τ2-bench** (agentic tool use benchmark) — it is highly capable at tool calling when served by other providers.

### Logs (redacted)

**Workers AI — Gemma 4 26B (`@cf/google/gemma-4-26b-a4b-it`):**
```json
{
"toolCalls": [
{"name": "list_items", "arguments": {}},
{"name": "lookup_item", "arguments": {"identifier": "some-item-slug"}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}}
]
}
```

**Workers AI — GLM 4.7 Flash (`@cf/zai-org/glm-4.7-flash`):**
```json
{
"toolCalls": [
{"name": "list_items", "arguments": {}},
{"name": "lookup_item", "arguments": {"identifier": "some-item-slug"}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}},
{"name": "perform_action", "arguments": {"itemId": "<|\"|", "quantity": 1}}
]
}
```

**OpenRouter (Parasail) — Same Gemma 4 26B model:**
```json
{
"toolCalls": [
{"name": "list_items", "arguments": {}},
{"name": "lookup_item", "arguments": {"identifier": "some-item-slug"}},
{"name": "perform_action", "arguments": {"itemId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "quantity": 1}}
]
}
```

### Environment

- **API endpoint:** `https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions` (OpenAI-compatible)
- **Models tested:** `@cf/google/gemma-4-26b-a4b-it`, `@cf/zai-org/glm-4.7-flash`
- **Client library:** `@mariozechner/pi-ai` v0.65.0 / `@mariozechner/pi-agent-core` v0.65.0 ([pi-mono](https://github.com/badlogic/pi-mono))
- **API protocol:** OpenAI-compatible `/v1/chat/completions` with tool definitions as JSON Schema (via TypeBox)
- **Runtime:** Cloudflare Worker with Durable Objects
- **Wrangler:** v4.71.0, `compatibility_date: 2026-03-12`, `compatibility_flags: ["nodejs_compat"]`
- **Worker placement:** `aws:us-east-2`
- **Date:** 2026-04-07

### Expected behavior

The model should reproduce the UUID from the tool result verbatim in subsequent tool call arguments.

### Actual behavior

The model emits a broken control token `<|\"|` — deterministic and identical across all models tested on Workers AI.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez par reproduire le problème sur le Workers AI OpenAI-compatible /v1/chat/completions endpoint avec @cf/google/gemma-4-26b-a4b-it et @cf/zai-org/glm-4.7-flash, en utilisant le flux de tool-call et la configuration JSON Schema décrits ici. Comparez les arguments obtenus avec le résultat d’OpenRouter ; c’est terminé lorsque les UUIDs sont reproduits verbatim au lieu du control token déterministe défectueux.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Domaine
ai, api, backend
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Calme
Clarté
À clarifier
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.