Workers AI OpenAI-compatible streaming: `delta.content` sometimes arrives as a JSON number instead of a string
- Dominant language
- TypeScript
- Stars
- 1.2k
- Forks
- 345
- Avg merge
- 13h 31m
- Merged PRs (30d)
- 1
Description
**Endpoint:** `POST https://api.cloudflare.com/client/v4/accounts//ai/v1/chat/completions` with `stream: true`
**Model:** `@cf/meta/llama-3.3-70b-instruct-fp8-fast`
**Seen:** 2 September 2026
I hit this running [recourse](https://github.com/ibrahimhajjaj/recourse), a customer support agent that learns your own content, against this endpoint from a Worker.
Some streaming chunks carry `choices[0].delta.content` as a JSON number rather than a string. Here is one, copied out of my Worker's logs exactly as it arrived:
```json
{"id":"id-1788378160014","created":1788378160,"model":"@cf/meta/llama-3.3-70b-instruct-fp8-fast","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":6}}],"usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1,"prompt_tokens_details":{"cached_tokens":0},"neurons":0.20480532944202423}}
```
The OpenAI streaming format has that field as a string, so a client that checks it against the spec rejects the chunk.
## What it looks like from the outside
Answers quietly lose characters, and then stop. A citation marker `[1]` comes out as `[]`, because the `1` is its own token. A price of `$0.005` comes out as `$`. After a few of those the stream dies part way through a sentence.
It took me a while to work out this was not my bug, because the visible symptom is missing text rather than an error.
## What I think is going on
The token text looks like it goes through a JSON parse somewhere and gets re-serialised as whatever it parsed to:
```
"6" -> 6
"0.005" -> 0.005
"155724\n" -> 155724 (the newline is eaten)
"005" -> parse fails, so it stays "005"
"$0.005" -> parse fails, so it stays "$0.005"
```
That accounts for every value I have seen, and it explains why only some numeric-looking text breaks while the rest is fine.
## Probably the same defect as #277
#277 reports AutoRAG streaming emitting numeric JSON for `response` chunks on this same model, with the values `155724` and `3e+50` and lost newlines. Different field, different surface, same coercion. That one has been open since September 2025. If the parse sits upstream of the OpenAI-compatible mapping, one fix covers both.
## Who this hits
- **Vercel AI SDK** through `@ai-sdk/openai-compatible`: throws `AI_TypeValidationError`, `expected "string"`, `path ["choices",0,"delta","content"]`, and the answer stops mid-stream.
- **Your own `workers-ai-provider@4.0.0`**: does not throw. It tests `textDelta.length > 0`, which is `undefined > 0` for a number, so the token is dropped without a word. Silent character loss rather than a crash.
## Reproducing
Stream any completion whose text contains a standalone number. Prompts about prices or numbered references hit it most often. The bad frames are intermittent, so it can take a few turns.
```bash
curl -N "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/v1/chat/completions" \
-H "Authorization: Bearer $CLOUDFLARE_AI_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"model": "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
"stream": true,
"messages": [{"role": "user", "content": "Reply with exactly: [1] the price is $0.005 a minute."}]
}' | grep '"content":[^"]'
```
Any line that comes back has an unquoted value in `content`.
## What I am doing until it is fixed
Every client on this endpoint has to put the value back before its own validation runs, so I ended up shipping a small `fetch` wrapper that rewrites `delta.content` to a string whenever it arrives as a number or a boolean: [`repairNumericContent`](https://github.com/ibrahimhajjaj/recourse/blob/4deb6ec/packages/core/src/models.ts#L157-L229). That keeps the digits and stops the stream dying, and it is safe precisely because a compliant server never puts a number there.
It does nothing for the whitespace, though. By the time the chunk reaches me the newline after `155724` is already gone, so that half is only fixable at your end.
## Expected
`delta.content` is always a string, and the token text reaches the wire as it left the tokeniser, whitespace included.
Contributor guide
Research direction
Start by running the supplied curl reproduction against the chat completions endpoint and inspect the streamed chunks for unquoted numeric content. Compare the behavior with related issue #277 and trace the OpenAI-compatible streaming response path. Done means delta.content is always a string and numeric-looking token text, including whitespace, is preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100