aws / aws/sagemaker-huggingface-inference-toolkit

Adjust request body data structure for numpy.ndarray for pipeline.

Open
#73 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
270
Forks
60
PR merge metrics
No merged PRs in 30d

Description

It's related to https://github.com/huggingface/transformers/issues/19743.

I think converting logic should be implemented.

In this toolkit's implementation, request body type should be json dict, but many pipelines does not accept list but np.ndarray. With SageMaker's serializer we cannot realize that.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the request-body handling and SageMaker serializer behavior described in the issue, then read the linked Transformers issue for the related pipeline expectations. Done means the toolkit can convert request data into the structure required by pipelines that accept numpy.ndarray rather than lists.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, numpy, python
Domain
api, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.