aws / aws/sagemaker-pytorch-inference-toolkit
How do I access Custom Attributes from the model during inference?
- Dominant language
- Python
- Stars
- 143
- Forks
- 73
- PR merge metrics
- No merged PRs in 30d
Description
**What did you find confusing? Please describe.**
It is clear how to [invoke an endpoint with Custom Attributes ](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime.html#SageMakerRuntime.Client.invoke_endpoint) but it is completely unclear to me how one would access this information from the Pytorch SDK inference code. I can see for the Tensorflow SDK they have documentation for it and imply the [second argument of the input handler function will provide access to it](https://github.com/aws/sagemaker-tensorflow-serving-container/blob/master/README.md#prepost-processing) but it seems in the Pytorch framework it is just a string indicating the input content MIME type.
It would seem from the [source ](https://github.com/aws/sagemaker-inference-toolkit/blob/7fcb80561c0d87cf035e194a86f9fa5c6c9d9b33/src/sagemaker_inference/transformer.py#L126)that the `context` object is not passed to the transformation functions, so not accessible at all to the user.
**Describe how documentation can be improved**
Explanation of how to access provided custom attributes using the Pytorch Inference Toolkit
**Additional context**
I want to achieve what is being done here https://aws.amazon.com/marketplace/pp/prodview-5jlvp43tsn3ny?sr=0-1&ref_=beagle&applicationId=AWSMPContessa - namely the ability to provide a confidence threshold to an inference request. I would like to use a binary format for the body of the payload (image/jpeg) so I cannot provide parameters as part of the body itself like if the body were a JSON.
Contributor guide
Research direction
Read src/sagemaker_inference/transformer.py at the linked transformation flow, then compare it with the linked TensorFlow serving documentation and the invoke_endpoint entry point. Document whether PyTorch input handlers can access Custom Attributes, including the supported way to provide them or the current limitation, and make the expected inference behavior unambiguous.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100