aws / aws/sagemaker-huggingface-inference-toolkit
Sagemaker endpoint inference Fails when following a tutorial
- Dominant language
- Python
- Stars
- 270
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I have been trying to use a Llava endpoint in sagemaker when following this tutorial (https://github.com/aws-samples/multimodal-rag-on-slide-decks/blob/main/Blog1-TitanEmbeddings-LVM/notebooks/0_deploy_llava.ipynb).
I havent used the Cloudformation, I am just interested in deploying the model itself.
The endpoint is deployed however, when I try to invoke it i get the following error:
ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
"code": 400,
"type": "InternalServerException",
"message": "GET was unable to find an engine to execute this computation"
}
Not quite sure what to do and what the exact issue is.
Contributor guide
Research direction
Start by reproducing the deployment and endpoint invocation in the 0_deploy_llava.ipynb tutorial, without the CloudFormation setup. Inspect the SageMaker inference configuration and the endpoint logs around the reported engine error. Done means identifying the configuration or compatibility cause and documenting or verifying a working model invocation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100