aws / aws/sagemaker-python-sdk
Create Awaitable predict capability
- 主要语言
- Python
- 星标
- 2.3k
- 派生
- 1.3k
- 平均合并
- 1 天 22 小时
- 30 天内合并 PR
- 35
描述
**Describe the feature you'd like**
Like many other inference libraries in python (e.g. OpenAI), create a real awaitable version of Predict for realtime sagemaker inference endpoints. This will help python applications that use FastAPI and asyncio to deliver realtime responses while not blocking the main event loop. Please note that this feature is different that the one currently available [here](https://sagemaker.readthedocs.io/en/v2.169.0/api/inference/predictor_async.html) where the predictions are written to a S3 bucket. This feature would work exactly like https://sagemaker.readthedocs.io/en/stable/api/inference/predictors.html#sagemaker.predictor.Predictor.predict but with an `await` in real `asyncio` style.
Sagemaker is an amazing library and it would be just way better for production environments using FastAPI to have this feature.
**How would this feature be used? Please describe.**
In this case, currently, the sync version looks like this:
```python
response = predictor.predict(input_data)
```
The async might be looking like
```python
response = await predictor.apredict(input_data)
```
**Describe alternatives you've considered**
I considered subclassing the `predictor` and add the async version.
**Additional context**
For modern python applications building on top of FastAPI and Asyncio, it is crucial to use async modalities do avoid blocking the main event-loop in the server (in case of scalable applications). Therefore, having a real `awaitable` functionality would avoid blocking the main event loop of the applications that leverage sagemaker.
Thanks alot
贡献指南
调研方向
从已记录的 Predictor.predict 行为开始,并将其与 issue 中链接的现有 predictor_async 功能进行比较,该功能会将预测写入 S3。跟踪实时 SageMaker 端点调用的执行方式,并确定可 await 等效功能所需的接口。完成标准是 Python asyncio 应用可以 await 实时预测,而无需使用基于 S3 的异步行为,也不会阻塞事件循环。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- aws, python
- 领域
- cloud, machine-learning
- Issue 类型
- 功能
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100