cloudwego / cloudwego/eino-ext

openai: multiline SSE data events fail with unexpected end of JSON input

Open
#925 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
811
Forks
368
Avg merge
16h 22m
Merged PRs (30d)
13

Description

**Describe the bug**

The OpenAI ChatModel fails to consume a valid Server-Sent Events event when one JSON payload is split across multiple `data:` lines. It tries to unmarshal the first `data:` line as a complete JSON value and returns:

```text
failed to receive stream chunk: unexpected end of JSON input
```

The SSE specification joins consecutive `data:` lines in the same event before dispatching the event. This matters for OpenAI-compatible gateways that serialize one JSON chunk over multiple `data:` fields.

**To Reproduce**

Run the following program:

```go
package main

import (
"context"
"fmt"
"io"
"net/http"
"net/http/httptest"

einoopenai "github.com/cloudwego/eino-ext/components/model/openai"
"github.com/cloudwego/eino/schema"
)

func main() {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
_, _ = io.WriteString(w, "data: {\"id\":\"chatcmpl-test\",\n"+
"data: \"object\":\"chat.completion.chunk\",\"created\":1,\"model\":\"test\",\n"+
"data: \"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"hello\"},\"finish_reason\":\"stop\"}]}\n\n"+
"data: [DONE]\n\n")
}))
defer server.Close()

chatModel, err := einoopenai.NewChatModel(context.Background(), &einoopenai.ChatModelConfig{
APIKey: "test", BaseURL: server.URL, Model: "test",
})
if err != nil {
panic(err)
}

stream, err := chatModel.Stream(context.Background(), []*schema.Message{schema.UserMessage("hello")})
if err != nil {
panic(err)
}
defer stream.Close()

_, err = stream.Recv()
fmt.Printf("%v\n", err)
}
```

Actual output:

```text
failed to receive stream chunk: unexpected end of JSON input
```

**Expected behavior**

The client should combine all `data:` lines belonging to the same SSE event, unmarshal the completed JSON payload, and return the assistant chunk containing `hello`.

**Version:**

- `github.com/cloudwego/eino-ext/components/model/openai v0.1.13`
- `github.com/cloudwego/eino-ext/libs/acl/openai v0.1.17`
- The current `main` branch is also affected as of 2026-07-18.

**Environment:**

```text
go version go1.26.4 linux/amd64
GOOS=linux
GOARCH=amd64
```

**Additional context**

The current ACL implementation depends on `github.com/meguminnnnnnnnn/go-openai v0.1.2`. Its stream reader calls `ReadBytes('\n')`, removes the `data:` prefix, and unmarshals that individual line immediately. It therefore does not implement the multi-line `data:` aggregation required by the SSE event model.

The dependency comment already describes this fork as a temporary solution and mentions a future switch to `github.com/openai/openai-go`. Either aggregating a complete SSE event in the current dependency path or migrating to the official SDK would address this case.

SSE specification: https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation

Contributor guide

Open the contributing guide

Research direction

Start at the OpenAI ChatModel Stream entry point and the ACL stream reader described in the issue, especially its ReadBytes('\n') handling. Run the provided httptest reproduction and trace how consecutive data: lines are processed. Done means the complete SSE event is unmarshaled successfully and stream.Recv() returns the assistant chunk containing "hello".

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ai, backend-api-design
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.