getsentry / getsentry/sentry-python

Create Ray Serve integration

未關閉
#3,231 0 則留言 8 個 reaction 已指派 0 人 在 GitHub 檢視
Feature New Integration Python
主要語言
Python
星號
2.2k
分支
669
平均合併
1 天 1 小時
30 天內合併 PR
213

描述

### Problem Statement

[From Ray Serve's docs](https://docs.ray.io/en/latest/serve/index.html):

> Ray Serve is a scalable model serving library for building online inference APIs. Serve is framework-agnostic, so you can use a single toolkit to serve everything from deep learning models built with frameworks like PyTorch, TensorFlow, and Keras, to Scikit-Learn models, to arbitrary Python business logic. It has several features and performance optimizations for serving Large Language Models such as response streaming, dynamic request batching, multi-node/multi-GPU serving, etc.

We should create a Ray Serve integration (ideally both tracing and errors). This integration was requested during a customer call with @leokster and @smeubank ([internal call notes](https://www.notion.so/sentry/Kaiko-ai-0b4c4d11e2794d358ec7dd2fdb13dcd5?pvs=4)), and it would help enhance our LLM Monitoring offering.

### Solution Brainstorm

Ray [integrates with OTel](https://docs.ray.io/en/latest/ray-observability/user-guides/ray-tracing.html), so we might want to wait until #2251 is implemented so we can use OTel for this integration. However, the Ray's OTel integration is no longer actively maintained, so we might need to fork it and maintain it ourselves.

Related: #2400 (Ray Remote integration)

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。