aws / aws/sagemaker-python-sdk

need to be able to set return data size for each request in batch transform

クローズ
#1,882 コメント 13 件 リアクション 1 件 担当者 1 名 @mollyheamazon が担当を希望しています GitHub で見る
component: Inference APIs and Interfaces type: feature request
主要言語
Python
スター
2.3k
フォーク
1.3k
平均マージ
1日 22時間
マージ済み PR(30日)
35

説明

**Describe the feature you'd like**
I'm running a NLP inference job to get sentence embedding vector for each record. Each record is less than 512 words only and the returned vector has 768 floats. Even if I set max_payload to 1 and max_concurrent_transforms to 1, I still got:
**io.netty.handler.codec.CorruptedFrameException: Message size exceed limit: 113038147**

I know there is a 5MB limit for endpoint inference. I'm relatively sure that this is caused by the size limit of return results of each request from batch transform, though I'm not able to find it in the doc.

I hope you can
1. Clarify the size limit for each request for batch transform in the doc.
2. Mimicking max_payload, expose a parameter to let user control the return data size for each request for batch transform.

**How would this feature be used? Please describe.**

Add a max_return_payload parameter to model.transformer like below.

tfm = model.transformer(instance_count=1,
instance_type='ml.p3.2xlarge',
accept='text/json',
assemble_with='Line',
output_path=batch_out,
strategy='MultiRecord',
max_payload=3,
**max_return_payload=30**)

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

Use strategy='SingleRecord' can bypass the issue. But it is significant slower as it's not making use of parallelism. Setting max_concurrent_transform to a larger value can make use of parallelism. But it could cause problem for code that's not designed for concurrency.

**Additional context**
Add any other context or screenshots about the feature request here.

A similar issue is raised here by other people: https://github.com/aws/sagemaker-python-sdk/issues/1096

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。