cloudwego / cloudwego/eino-ext

openrouter: buildResponseChunkMessageModifier panics on the nil msg documented for end=true

Open Beginner friendly
#999 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
811
Forks
368
Avg merge
16h 22m
Merged PRs (30d)
13

Description

**Describe the bug**

`ResponseChunkMessageModifier` documents that its message may be nil when `end` is true — [libs/acl/openai/option.go#L46](https://github.com/cloudwego/eino-ext/blob/main/libs/acl/openai/option.go#L46):

```go
// ResponseChunkMessageModifier transforms the generated message chunk using the raw response body.
// When end is true, msg and rawBody may be nil.
type ResponseChunkMessageModifier func(ctx context.Context, msg *schema.Message, rawBody []byte, end bool) (*schema.Message, error)
```

openrouter's modifier dereferences it on its first line — [components/model/openrouter/chatmodel.go#L353](https://github.com/cloudwego/eino-ext/blob/main/components/model/openrouter/chatmodel.go#L353):

```go
if msg.ResponseMeta != nil && msg.ResponseMeta.FinishReason == reasonError {
```

`msg` is the nil. The stream loop is doing what the comment says it may do, so this is the component's side of the contract rather than `acl/openai`'s. The three other modifiers in the repo — `agenticopenai`, `agenticdeepseek`, `agenticqwen` — all check `msg != nil && msg.ResponseMeta != nil`. This one has the second half and not the first.

The nil arrives whenever the last chunk of the stream carried content. `lastEmptyMsg` is only assigned from a chunk that built an empty message, and any chunk with content clears it — [libs/acl/openai/chat_model.go#L928](https://github.com/cloudwego/eino-ext/blob/main/libs/acl/openai/chat_model.go#L928):

```go
if msg.Content == "" && len(msg.ToolCalls) == 0 && !(ok && len(rc) > 0) {
lastEmptyMsg = msg
continue
}

lastEmptyMsg = nil
```

So it does not take a malformed stream. A provider that puts `finish_reason` on its last content delta rather than in a separate empty chunk produces one, and `finish_reason` is a per-choice field on every chunk, so that stream is well formed. OpenAI and OpenRouter both send the separate empty chunk, which is why this stays quiet against them.

The reply is lost after it has already arrived in full. The stream goroutine recovers the panic and forwards it as an error, so the process survives and the caller gets a stack where the text should be.

**To Reproduce**

`httptest` stands in for the provider, so no key is needed. Three endings against `openrouter@v0.1.10`:

| the stream ends with | result |
| --- | --- |
| an empty delta plus `finish_reason` — what OpenAI and OpenRouter send | ok |
| `finish_reason` on the last content chunk | panic |
| nothing after the last content chunk | panic |

```go
package repro

import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"testing"

"github.com/cloudwego/eino-ext/components/model/openrouter"
"github.com/cloudwego/eino/schema"
)

func delta(content string, finish any) map[string]any {
choice := map[string]any{"index": 0, "delta": map[string]any{"role": "assistant", "content": content}}
if finish != nil {
choice["finish_reason"] = finish
}
return map[string]any{"id": "c1", "object": "chat.completion.chunk", "created": 1, "model": "m", "choices": []any{choice}}
}

func emptyFinal() map[string]any {
return map[string]any{"id": "c1", "object": "chat.completion.chunk", "created": 1, "model": "m",
"choices": []any{map[string]any{"index": 0, "delta": map[string]any{}, "finish_reason": "stop"}},
"usage": map[string]any{"prompt_tokens": 7, "completion_tokens": 4, "total_tokens": 11}}
}

func stream(t *testing.T, chunks ...map[string]any) (string, error) {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
f := w.(http.Flusher)
for _, c := range chunks {
b, _ := json.Marshal(c)
fmt.Fprintf(w, "data: %s\n\n", b)
f.Flush()
}
fmt.Fprint(w, "data: [DONE]\n\n")
f.Flush()
}))
defer srv.Close()

ctx := context.Background()
cm, err := openrouter.NewChatModel(ctx, &openrouter.Config{
APIKey: "not-a-real-key", Model: "m", BaseURL: srv.URL + "/v1",
})
if err != nil {
t.Fatalf("NewChatModel: %v", err)
}
sr, err := cm.Stream(ctx, []*schema.Message{schema.UserMessage("say it")})
if err != nil {
return "", err
}
defer sr.Close()
var out string
for {
msg, err := sr.Recv()
if err == io.EOF {
return out, nil
}
if err != nil {
return out, err
}
out += msg.Content
}
}

// What OpenAI and OpenRouter send: a separate final chunk with an empty delta.
func TestEmptyFinalChunk(t *testing.T) {
out, err := stream(t, delta("a", nil), delta("b", nil), emptyFinal())
t.Logf("out=%q err=%v", out, err)
}

// finish_reason on the last content chunk. Panics.
func TestFinishReasonOnLastContentChunk(t *testing.T) {
out, err := stream(t, delta("a", nil), delta("b", "stop"))
t.Logf("out=%q err=%v", out, err)
}

// Nothing after the last content chunk. Panics.
func TestNoFinalChunk(t *testing.T) {
out, err := stream(t, delta("a", nil), delta("b", nil))
t.Logf("out=%q err=%v", out, err)
}
```

`out="ab"` in the two failing cases is the whole reply, received and then dropped:

```
=== RUN TestEmptyFinalChunk
repro_test.go:73: out="ab" err=
--- PASS: TestEmptyFinalChunk (0.00s)
=== RUN TestFinishReasonOnLastContentChunk
repro_test.go:79: out="ab" err=panic error: runtime error: invalid memory address or nil pointer dereference,
stack: goroutine 63 [running]:
runtime/debug.Stack()
github.com/cloudwego/eino-ext/libs/acl/openai.(*Client).Stream.func2.1()
libs/acl/openai@v0.1.17/chat_model.go:868 +0x60
panic({0x104e961a0?, 0x104fd5f50?})
github.com/cloudwego/eino-ext/components/model/openrouter.(*ChatModel).buildOptions.(*ChatModel).buildResponseChunkMessageModifier.func11({0x104f68008?, 0x104fd5d60?}, 0x0, {0x0, 0x0, 0x0}, 0x0?)
components/model/openrouter@v0.1.10/chatmodel.go:353 +0x28
github.com/cloudwego/eino-ext/libs/acl/openai.(*Client).Stream.func2({0x104f6feb0, 0x6cbbddcab470})
libs/acl/openai@v0.1.17/chat_model.go:881 +0x81c
created by github.com/cloudwego/eino-ext/libs/acl/openai.(*Client).Stream in goroutine 20
libs/acl/openai@v0.1.17/chat_model.go:862 +0x3e4
```

`0x0` in the openrouter frame is the nil `msg`. `TestNoFinalChunk` gives the same stack.

**Expected behavior**

A stream whose last chunk carries content delivers its reply, the same as one that ends with an empty delta.

Returning the nil straight back is already handled by the caller: it assigns the result to `lastEmptyMsg` and only sends when that is non-nil. So the guard the other three components have looks like the whole fix.

```diff
return func(ctx context.Context, msg *schema.Message, rawBody []byte, end bool) (*schema.Message, error) {
const reasonError = "error"

+ if msg == nil {
+ return msg, nil
+ }
+
if msg.ResponseMeta != nil && msg.ResponseMeta.FinishReason == reasonError {
```

All three cases above return `"ab"` with that applied. Happy to send it as a PR against `develop` if you would like it that way.

**Version:**

- `components/model/openrouter` v0.1.10
- `libs/acl/openai` v0.1.17
- `eino` v0.9.19

Newest released for each, and both lines are unchanged on `main` today.

**Environment:**

`go1.26.5`, `GOOS=darwin`, `GOARCH=arm64`.

**Additional context**

Found with a test server that ended the stream after the last content chunk. The `finish_reason`-on-the-last-chunk row is the one that looks reachable against a real provider, which is why it is in the table.

Contributor guide

Open the contributing guide

Research direction

Start in components/model/openrouter/chatmodel.go at buildResponseChunkMessageModifier and compare its nil handling with the agenticopenai, agenticdeepseek, and agenticqwen modifiers. Use the provided httptest reproduction for streams ending with content, then verify the full reply is returned without an error and that the existing empty-final-chunk case remains successful.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.