aws / aws/sagemaker-huggingface-inference-toolkit
How to access CustomAttributes in async inferece request input_fn
- Dominant language
- Python
- Stars
- 270
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
I need to access [CustomAttributes](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpointAsync.html) in async request.
I tried to access it inside a custom `input_fn`
`def input_fn(self, input_data, content_type):`
self.context coud give access to custom [properties](https://github.com/aws/sagemaker-huggingface-inference-toolkit/blob/main/src/sagemaker_huggingface_inference_toolkit/handler_service.py#LL203C28-L203C35)
but I am not sure about this
```
try:
if not self.initialized:
self.initialize(context)
```
(I am new to python). But this looks like it would instantiate the class once and fill the context once, so it would not get update on each request with the additional context params i guess?!
anyhow. it looks like `self`
will not be accessible inside
`def input_fn(self, input_data, content_type):`
hence no access to additional params.
@philschmid Any idea about how to access additional CustomAttributes ?
Contributor guide
Research direction
Read src/sagemaker_huggingface_inference_toolkit/handler_service.py around context initialization, then trace how the async request reaches the custom input_fn(input_data, content_type) entry point. Determine whether CustomAttributes are refreshed per request; done means the issue has a documented and tested access path, or a clear explanation that the current hook cannot expose them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- api, backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100