schema: ConcatMessages panics or drops data when a base64 audio stream is followed by a non-base64 audio part
- Dominant language
- Go
- Stars
- 13k
- Forks
- 1.1k
- Avg merge
- 4h 6m
- Merged PRs (30d)
- 41
Description
## Summary
When concatenating streamed `AssistantGenMultiContent` chunks, a group of base64 audio parts absorbs **any** later `audio_url` part with the same streaming index, because `canMergeOutputParts` only validates the mergeability of the group head, not the part being appended. This produces two failure modes:
1. **Panic (SIGSEGV)**: if the trailing part has `Audio == nil`, `mergeAudioParts` dereferences the nil `Audio` pointer (`schema/message.go:1526`), crashing the process during stream aggregation.
2. **Silent data loss**: if the trailing part is a URL-referenced audio (`Audio.URL` set, no `Base64Data`), it is merged into the base64 group; its URL is dropped entirely and MIME metadata is mixed across chunks.
## To Reproduce
```go
package schema
import "testing"
func strptr(s string) *string { return &s }
// panics with nil pointer dereference
func TestAudioNilPanic(t *testing.T) {
_, _, _ = ConcatMessages([]*Message{
{Role: Assistant, AssistantGenMultiContent: []MessageOutputPart{
{Type: ChatMessagePartTypeAudioURL, Audio: &MessageOutputAudio{
MessagePartCommon: MessagePartCommon{Base64Data: strptr("QUJD")}}}},
}},
{Role: Assistant, AssistantGenMultiContent: []MessageOutputPart{
{Type: ChatMessagePartTypeAudioURL, Audio: nil},
}},
})
}
// the URL part is silently swallowed: result has 1 part, URL lost
func TestAudioURLDropped(t *testing.T) {
m, _ := ConcatMessages([]*Message{
{Role: Assistant, AssistantGenMultiContent: []MessageOutputPart{
{Type: ChatMessagePartTypeAudioURL, Audio: &MessageOutputAudio{
MessagePartCommon: MessagePartCommon{Base64Data: strptr("QUJD")}}}},
}},
{Role: Assistant, AssistantGenMultiContent: []MessageOutputPart{
{Type: ChatMessagePartTypeAudioURL, Audio: &MessageOutputAudio{
MessagePartCommon: MessagePartCommon{URL: strptr("https://x.com/a.wav")}}},
}},
})
// len(m.AssistantGenMultiContent) == 1, the URL part is gone
_ = m
}
```
## Root cause
`groupOutputParts` calls `canMergeOutputParts(currentGroup[0], parts[i])`, and `isMergeableOutputPartType` is only evaluated for `currentGroup[0]`:
```go
if !isMergeableOutputPartType(current) { // next is never checked
return false
}
```
A group headed by a base64 audio part is mergeable, so every subsequent `audio_url` part (nil payload, URL payload, ...) joins the same group regardless of its own mergeability.
## Expected behavior
- A trailing nil/URL audio part should not join a base64 audio merge group; it should be preserved as its own part in the concatenated message.
- `ConcatMessages` must never panic on malformed part sequences.
## Version
Current main (`9d983b36`) and `alpha/10`.
## Proposed fix
Check `isMergeableOutputPartType(next)` as well in `canMergeOutputParts`, and return an error from `mergeAudioParts` for non-base64 parts as a defensive measure. I have the fix with regression tests ready and will open a PR shortly.
Contributor guide
Research direction
Start in schema/message.go, especially canMergeOutputParts, groupOutputParts, and mergeAudioParts; reproduce the panic and dropped URL with the supplied test cases. Add regression tests for nil and URL audio parts, then verify concatenation preserves them as separate parts without panicking. The reporter says they already have a fix and plan to open a PR, so check whether that work is underway before starting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100