getsentry / getsentry/sentry-javascript

Reducing memory foot print for vercel AI messages

Đang mở
#21,235 4 bình luận 0 reaction 0 người được giao Xem trên GitHub
Feature
Ngôn ngữ chính
TypeScript
Star
8.7k
Fork
1.8k
Merge trung bình
1 ngày 17 giờ
Pull request đã merge (30 ngày)
515

Mô tả

### Problem Statement

When `sendDefaultPii: true`, [requestMessagesFromPrompt](https://github.com/getsentry/sentry-javascript/blob/72346740a44620368fe07478362cd272d14abe3d/packages/core/src/tracing/vercel-ai/utils.ts#L258) parses and re-serializes `ai.prompt.messages` on every AI span, even when no transformation is needed.

For a 4mb prompt, this briefly doubles memory usage and has inside the function another memory footprint (which is unavoidable I guess):
1. Original JSON string on the span attribute
2. Parsed array from `JSON.parse`
3. Re-serialized string from `getJsonString`/`getTruncatedJsonString`

The parse + reserialize is only necessary when:
- System instructions need to be extracted (messages contain role: "system")
- Truncation is enabled

In the common case (no system messages, no truncation), we could skip parsing entirely and reuse the original string.

### Solution Brainstorm

Not sure if there is a good solution for it or if it is worth fixing as 4mb is unusually big, but if there is a high traffic this could spike real fast.

### Additional Context

_No response_

### Priority

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding `+1` or `me too`, to help us triage it.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.