aws / aws/sagemaker-python-sdk

Create Awaitable predict capability

未关闭
#3,973 10 条评论 5 个 reaction 已指派 0 人 在 GitHub 查看
component: async inference type: feature request
主要语言
Python
星标
2.3k
派生
1.3k
平均合并
1 天 22 小时
30 天内合并 PR
35

描述

**Describe the feature you'd like**
Like many other inference libraries in python (e.g. OpenAI), create a real awaitable version of Predict for realtime sagemaker inference endpoints. This will help python applications that use FastAPI and asyncio to deliver realtime responses while not blocking the main event loop. Please note that this feature is different that the one currently available [here](https://sagemaker.readthedocs.io/en/v2.169.0/api/inference/predictor_async.html) where the predictions are written to a S3 bucket. This feature would work exactly like https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html#sagemaker.predictor.Predictor.predict but with an `await` in real `asyncio` style.

Sagemaker is an amazing library and it would be just way better for production environments using FastAPI to have this feature.

**How would this feature be used? Please describe.**
In this case, currently, the sync version looks like this:
```python
response = predictor.predict(input_data)
```
The async might be looking like
```python
response = await predictor.apredict(input_data)
```

**Describe alternatives you've considered**
I considered subclassing the `predictor` and add the async version.

**Additional context**
For modern python applications building on top of FastAPI and Asyncio, it is crucial to use async modalities do avoid blocking the main event-loop in the server (in case of scalable applications). Therefore, having a real `awaitable` functionality would avoid blocking the main event loop of the applications that leverage sagemaker.

Thanks alot

贡献指南

打开贡献指南

调研方向

从已记录的 Predictor.predict 行为开始,并将其与 issue 中链接的现有 predictor_async 功能进行比较,该功能会将预测写入 S3。跟踪实时 SageMaker 端点调用的执行方式,并确定可 await 等效功能所需的接口。完成标准是 Python asyncio 应用可以 await 实时预测,而无需使用基于 S3 的异步行为,也不会阻塞事件循环。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, python
领域
cloud, machine-learning
Issue 类型
功能
难度
5/5
预计耗时
一周以上
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。